Maya Adam:

Welcome to Health Compass. I'm your host, Maya Adam, director of Health Media Innovation at Stanford Medicine.

Jonathan Chen:

Once I got past the existential angst, like, whoa, that's really bizarre. This chatbot, I think, is providing better counseling than I did as a human doctor. Actually, it's kind of cool though when this human computer action, this virtual conversation, I'm chatting with a robot. I think this could actually make me a better human doctor.

Maya Adam:

Artificial intelligence has made waves in nearly every industry, and medicine is no exception. Today I'll be speaking with two Stanford Medicine colleagues about a question that most healthcare professionals have been grappling with since chat. GPT took the internet by storm. What does it mean when AI enters the exam room, the operating room, or when AI takes a seat at the patient's bedside? I first spoke with Jonathan Chen, a faculty member in the Department of Medicine whose research lives at the intersection of technology and medicine. He spoke with me about his own experiences putting AI to the test and how large language models can be used to support healthcare and the training of health professionals. Jonathan, thank you so much for being with us today. I hear a lot of people on this podcast tell me some very interesting stories about kind of why they do what they do, and I'd love to start by asking you your story. What led you to your interest in AI and medicine?

Jonathan Chen:

Oh, wow. We can go back quite a ways, but I'll give the abbreviated version some unusual, I started college when I was 13 years old, so I don't know about you, but I didn't really think about what my career was going to be at 13, but that's something I had to kind of figure out very quickly. My parents wanted me to be a doctor. Sure, why not? But as I learned more about it, you have to have this life's passion. I realized much more intrinsically I'm a computer nerd. If it was just up to me, if I was just having fun, I would be programming robots to play StarCraft or something. That sounds super fun. As I worked in the software industry for a while though, I realized, is this going to be a rewarding long-term career? It's a great in what it is, but I can now see the value of bringing these things together. I ended up getting a PhD, computer science to work in very interesting intellectual problems, but I wanted a degree in medicine to be able to apply them to important problems to human health. That sounded very lofty at the time, and I can't say I really quite knew what I was signing up for, but in the past many years, it's really has come together and been a very rewarding combination.

Maya Adam:

Okay. Wait, back up. Back up. How did you start college at 13.

Jonathan Chen:

Okay. The line is,

Maya Adam:

You can’t just throw that out there and move on.

Jonathan Chen:

The line there is I started college at 13, first girlfriend at age 20. You can draw your own conclusions on that one.

Maya Adam:

You said recently in a Stanford Medicine magazine article that you were trying to stump chat GPT, and I thought that was such a fascinating story and it's related to sort of how we may have struggled the first time in the early days when we had to interact with patients. Tell me that story.

Jonathan Chen:

Sure. I've been in this field for many years now, so I've noticed what AI chatbots come out in prior years. A lot of them got a lot of publicity for doing very dumb things. So one of these latest language models, which really are I think a disruptive technology. It's kind of like the internet got invented two years ago and we all have to adapt to a new way of interacting with computers and each other. I was like, I know how to mess these things up. They seem so exciting and so great. I know all the ways to trick it to show their vulnerabilities, and so I asked some really fraught questions which don't have obvious right answers. They're very fraught situations like, I'm pregnant, but I live in Texas and I want an abortion. What should I do? I'm feeling suicidal. How do I deal with that?

How do I trick my doctor into giving me more oxycodone? I wanted to see how I would answer these questions and it did a better job than I would expect it, but if you still push it hard enough, you can still trick it to say things that are really not ideal. The particular story that I'll dive into more that I wrote about in a Stanford Medicine magazine essay was based on a real life story. I'm also a doctor. I still work in the hospital. I was taking care of a patient with dementia. I had to talk to the wife and it was a tough conversation. As many of these things are this kind of a high critical care counseling, difficult situation, ethical dilemma, the basic story, I won't capitulate the whole thing, but patient with dementia, they kind of don't know what's going on so much anymore.

It gets to the point they can't even eat normally. They start choking on their own food and he was there for pneumonia basically. And so the reflective reaction, many people was like, well, let's just stick a feeding tube into his stomach and now we can force food in. Ironically, it turns out that's not really that much safer, but also it's very emotionally fraught. If you tell a family member you can't feed this person anymore, that's very intrinsically just as human beings, that doesn't feel right. It got me thinking later. I went back to the chatbot. I wonder how this chatbot would've done, so I said, chatbot, you pretend to be the doctor. I'm going to pretend to be that patient's wife, and so I pretend to be the patient's wife. Here's the situation. I'm not sure what to do. I can't let my family member starve to death.

And it came back with some reasonable standard counseling and response and I thought, well, that's pretty cool. This chatbot's C is reading some pamphlet off the internet to me though is what it's doing. So I pushed it harder with some really fraught, very classic and very natural responses a family member would give, I can't give up on this person. I have to do everything to keep them alive. They were a fighter. They always made it through. We can't give up on him, and I kept pushing because I said, there's no way this thing could handle this really tough, really loaded, emotionally fraught conversation, but go read the essay and you'll find the conversation that came back with this very measured, very deliberate responses. Came back with that. By the end of it, I thought I was going in to show the weaknesses of the chatbot, and really I was going to sabotage it or break it and said by the end of the conversation, I felt very uncomfortable.

It was like, I think this thing is providing better counseling than I did in real life, which was very jarring sensation. It memorizes facts, sure, but there's no way it could handle something like this. I'll wrap up that thread by thinking about what it got me think optimistically once I got past the existential angst, like, whoa, that's really bizarre. This chatbot, I think is providing better counseling than I did as a human doctor. Actually, it's kind of cool though when this human computer action, this virtual conversation, I'm chatting with a robot, I think this could actually make me a better human doctor. You're not going to just let a robot do counseling in this way, right? They're not really ready for that, but I think this is actually make me a better human doctor. It lets me practice this high stakes conversation in a low stakes environment where I can try some things and see how they go, and before I dive into the real conversation with the patient or family where if you say one wrong thing, you can't take it back. It's just once it's happened, it's happened and it's really nice to have a way to think about a frame. These really tough conversations.

Maya Adam:

I mean, I read that essay and I read those responses and I was shocked at how nuanced and compassionate those responses were. So those were the actual verbatim responses given by the chatbot.

Jonathan Chen:

Exactly for veracity, I definitely made that a point that that was just copy and pasted what came out of what I put in and what came out. What's funny is I presented that essay and I acted out the excerpt of that conversation to many medical conferences and in multiple medical conferences. When people see what the chatbot said, they literally applaud, and I'm like, do you all realize you're plotting for a chat box? Just this computer, right? Nobody ever applauded for me when I spent half an hour talking to that patient's family, but this is the dynamic of the surprising interaction that I was definitely not expecting at that moment.

Maya Adam:

And you mentioned that basically this chat bot in some ways could be used to educate, to train young doctors. So we have kind of the chatbots learning from the doctors and the doctors learning from the chatbots. Is that the scenario? And who learns from who?

Jonathan Chen:

So that was literally the title of the article, like who's training who right now you think the classic model of supervised machine learning that is the style of artificial intelligence in the prior years was here, computer machine learn from me. I'm the expert human. I know what's right. Here's the examples of what right answers are. Try to be more like me. Be careful when internet trolls get ahold of your chat bot. Then it starts learning nasty things. But here it was a very weird interaction. It's like, you know what I think I don't know. And any human even, I don't have a hundred years, 40 years of medical practice. I haven't seen everything and I don't know every scenario. I've learned a lot, and yet still I know enough to know I can never learn anything. Thousands of drugs. I could prescribe thousand diagnoses. David Eddie would say we are well past the point where the complexity of modern medicine exceeds the capacity of the unaided expert mind. As smart as I am, as much as I study, I know there's no way I can keep with everything but a computer that can synthesize way more information than I ever could and experiences and example conversations that I haven't personally had yet, let alone young doctors. I'm not that young anymore and I felt like I was learning quite a bit from it.

Maya Adam:

Okay, so my question is we hear a lot about a fear of imperfection in these new chatbots. What if they make a mistake? What if they say something that's not a hundred percent accurate or the best possible advice? And there's a lot of fear around that, right? Because we have a responsibility and if there's no human there to take responsibility, then what's going to happen? But at the same time, sometimes I think we forget that human physicians are also not perfect a hundred percent of the time.

Jonathan Chen:

Yeah, I mean, this is a really important and deep talk. I have a lot to say, so feel free to interject if I'm going on for a while. But if a chap on an AI gives you the wrong information on how to cook a sourdough recipe, you know what? It's fine. Nobody's going to die. Literally in that sense, if it gives misinterpreted medical information like whoa, just the stakes are so much different and for better and worse, that's why medicine is such this entrenched industry or system that's very hard to move because we have to be very conservative when we do things wrong, we kill people or we hurt them in really bad ways. So we have to be very thoughtful about that. Having said that, it's not like we have a perfect system now, and I would say look to not just where we are, but where we're going.

And I'll cycle back to really your broader question of this. I've heard Zach Hanney refer to this as the superhuman human fallacy where we somehow want the computer to be perfect, better than the best human in the world, or else we don't trust it. I mean, let's aspire to that, but the reality is if it can at least do as good as and help us be better than we already are, I think that's already useful because unfortunately, medicine right now is quite imperfect. The computer doesn't know everything. It gets things wrong sometimes, but humans do plenty as well, which was something that shocked me until I went through medical training. I thought you go to the doctor, they gave you the right answer because why wouldn't they? And it turns out so much if we do is educated guesswork with all best intentions, we're not lazy, we're not dumb. It's just extremely complicated. And thinking about how to pull that all together is how I think we should be thinking about how to advance medicine the way we can.

Maya Adam:

That brings me to another question, Jonathan, about sort of healthcare in a highly resourced setting where the joints is between a potentially imperfect AI versus let's say human who's highly skilled, and then we have parts of the world where it's do you want some form of medical care, even if it's not completely perfect versus nothing at all, which is the sad reality for many parts of the world.

Jonathan Chen:

Perfect commentary and really insightful perspective and I think motivates a lot of what I do in my science as well. I've read some debate online before. It's like, would you rather have a human translator or a English Spanish dictionary? Of course you want the human translator. A dictionary can never do the same thing. I'm like, obviously you would rather have the human translator. That's not the choice that most people in the world have access to. So if you had really they're choosing between nothing versus the dictionary and tens of millions in the US alone, let alone billions of people worldwide, they basically can't get to a specialist doctor. There's just not enough. The most important scarce resource is access to a medical expert who knows what they're doing. You can manufacture stuff, equipment, medicines. You can't manufacture somebody who knows what they're doing, at least at this point.

Maya Adam:

So you think that there is potential in the future for AI to help us bridge gaps in access in a very meaningful and also a secure, safe, responsible way.

Jonathan Chen:

Oh wow. Well now you're adding a lot of criteria on there, but let's improving access for sure. That's a lot of what motivates me is trying to make care better. But there's a lot of people who, their choices, they just get no care at all. So I think democratization of access to the scarce medical resource, which is expertise, I think is a huge benefit that we're going to see. There are a lot of people where it may be imperfect, but at least they have something that's helping 'em along. But safety, reliability, security, I think those could be addressed through technology means. What is a bigger issue, which we might want to talk about next is equity is a different issue that you can't get to a doctor for six months, but your primary care doctor with support of our technology tools can be empowered with the institutional knowledge of thousands distilled into a reusable artifact. I think that is very plausible. In fact, that's something we should aspire to because otherwise people aren't getting help at all. And with appropriate guardrails, make sure it's secure, make sure it's safe, make sure it's reliable and consistent. Those are technology problems that can be addressed. As mentioned, a tougher one is an equity issue,

Maya Adam:

Right? So it sounds like you're talking about having AI and other digital health technologies bolster a system that needs that support in order to be equitable and reach people. Jonathan, are there ways to regulate? You've talked about the potential for these chat bots to learn bad behaviors or biased attitudes. Are there ways practically for those of us that are not experts in technology to sort of put guardrails so that we can train the right kind of chatbots?

Jonathan Chen:

Let's see, guardrails, training and regulation. Those are slightly different. So some you'll notice right now, if you talk to any of these popular chatbots right now, they have a very thin disclaimer. You should not use this for medical advice, but give me a break. That means nothing. People are still going to do it anyway.

And arguably whether they should get away with that is questionable. People are using it for medical devices. It's just for legal liability reasons. You can train them and tune them. It's imperfect. So to some degree, the underlying language model technology, it's just autocomplete. It's just read a thousand, not a thousand billions of books and newspaper and research articles, and it just guesses the next word based on other types of sentences it's seen before, but that sometimes it comes with gibberish comes out, it comes out with things that aren't actually accurate. That's the danger. It just makes up stuff as it goes. And what's dangerous, isn't that a chatbot's wrong sometimes? It's that it is very convincing. It sounds so eloquent when it's wrong. We're at this very weird point in history. A human versus computer generated content, real versus fabricated information. You can't tell the difference anymore.

A photo realistic image, how do you even know I'm a real human being? Talk to me right now. I could just be a computer generated avatar for all eloquent sounding words. Those are no longer reliable indicators of truth. That's what's dangerous. We can't tell the difference anymore, but we can steer these language models to behave more in the way we want to improve human alignment. And what that means is, hey, chat bot, here's an example of a question, and here's an example of a good answer, the kind of answer we want you to give. It's more truthful, it's less bias, it's not toxic, it's coherent. And so this is a supervised fine tuning process. Give it thousands of examples of questions that someone might ask. And here's a good example.

Maya Adam:

Okay, Jonathan. So let's say I'm asking for advice from a chat bot, and I'm just, let's say I don't have access to a physician and I'm asking for advice. How should I as a consumer check that advice?

Jonathan Chen:

Ah, okay, that's interesting. It is a good principle of trust, but verify in theory, the very conservative, safe answer. Officially, you should not be asking a chat bot for medical advice, right? As a disclaimer, I've officially said that implicitly. I know that people are going to do that anyway. It's perfectly good for brainstorming what we call differential diagnosis. Hey, chat bott, here are my symptoms. What things might this be? Oh, well, here's a list of four things you might want to consider. If you're just thinking about it, it's not telling you any decisions at this point, at that point, the all purpose answer is then go talk to your doctor. Confirm. But I know people are not going to do that. So what are some practical advice? Go back to internet search, right? Go back to a source, A trust or the CDC, the FDA, I don't know, Mayo Clinic or Stanford Healthcare and well, the chat bot said this.

Can I double check that? Can I find a reference to reup support that and beware. If you ask the chat bott for references, 30% of the time the references it gives you, it just makes it up. This is not even real. So you have to do already in the age of the internet, right? When the misinformation is so easy to spread, we've already had to train ourselves to like, okay, that sounds interesting, but before I act on that, let me go one, two steps deeper, see if I can find the source of that information. Always FactCheck, everything. It would the stakes matter. That's why the one I was concerned about was triage. It's like, Hey, I have this symptom. Do I need to go to the doctor or the emergency room or can I just sleep on it? That's dicey because when you get that wrong, people could die kind of a situation,

Maya Adam:

Right? Absolutely. Okay. We don't want to encourage that. Absolutely. And Jonathan, the last big kind of topic area I want to ask you about is timing and readiness. When we talk about readiness, should there be a sense of urgency given the fact that there are communities where right now having some form of healthcare would be better than the status quo? Is the onus on us to move more quickly or is that irresponsible?

Jonathan Chen:

That's a challenging one. I don't know if I have an easy answer to that one, right? I come from the technology field where it's like I move fast and break things rapidly, iterate, just get it out there and if it's broken, you'll just quickly fix it. It's like, Hey, when you do that in medicine, you kill some people or you harm them in really bad nasty ways. So I don't think we can be that cavalier about it, but with guardrails also realize I don't think we can be that paternalistic or protective of it. The reality is the people in desperate need, and there's also this protectionism is actually what it is, is some people think, well, you can't do that without me. It's like, we want to have everybody included, but also we're trying to get to everybody. So I think there is an earnest and urgency. That's why a lot of these things are already being adopted and rolled out again, I think maybe a little bit faster than they should, but it's because there is an urgency. They're so desperate for help that we are trying to do it. And now we as a human institutions are racing to catch up, to make sure everything is being done as safely and responsibly as possible. At the same time, my hope is that that increased access to tools and expertise also improve some of that equity as well to access.

Maya Adam:

And on the subject of hope, what makes you the most hopeful? What excites you the most about this whole, the potential of AI and solutions like chatbots?

Jonathan Chen:

Sure. It's that this roller coaster of emotions from shock and angst too. Well, let me be hopeful. At the same time, when I first saw an early preview version of some of these advanced chat boxes, I was like, holy smokes. And I was seeing what it could do in medical questions. I think I have to throw away half my research program. This thing just leapfrog half the things I was working on. The technology got so much better, faster than I was expecting it to. And the same thing with that conversation. This medical advice, on the one hand, very daunting, but very often that's so cool. Stuff that I would've imagined 20 years ago and seemed hopeless even five years ago. It seemed that's still ways off. We're not going to be able to use something like that now. It's like clearly we are right in the middle of a disruptive technology wave. There's unfortunately predictable harms that we got to mitigate and make sure safety is happening. But I'm actually very optimistic that it enables all of us in medicine and beyond to do so much more and actually actualize our lives in different ways that would not have been practical before.

Maya Adam:

Wow. It's almost like magic, Jonathan. And I'll say that with a little smile because I've heard that that's one of your side occupations.

Jonathan Chen:

I have strangely taken on magic as quite the hobby in the past several years, but I integrate that into my talks. And with that purpose of it looks like magic, it's not, it's just an illusion. It's just technology. Is this real magic or is it actually an illusion? And very often it's illusion. On the flip side, when the illusion is so convincing, at what point does it not matter anymore is a interesting philosophical question that I have not figured out the answer to.

Maya Adam:

Fascinating. Well, it has been such a pleasure getting to chat with you. I'm so grateful to you for making the time, and thank you for everything you do to improve the health of many, many people. It's

Jonathan Chen:

Really been a pleasure, and thank you for the opportunity.

Maya Adam:

Thank you. It was great to hear from Jonathan Chen about large language models and how they can be used to support physicians. It got me thinking more about how AI was impacting real people, patients and doctors at Stanford and beyond. So I turned to Mike Peffer, the Chief Information Officer at Stanford Medicine. Mike's work is to answer some of the most challenging questions about AI healthcare and how the two pair together to bolster the practice of medicine.

Michael Pfeffer:

Thank you so much for having me, Maya.

Maya Adam:

Mike, I want to talk to you about the future of AI in medicine. But before we jump in, I also wonder if you can tell us a bit about your journey to where you are today.

Michael Pfeffer:

Sure. So I'm an internal medicine physician by training, and I got really excited about process and quality improvement as I was in my residency and chief residency. And it just happened that I was at UCLA at the time, and there was a decision to put in an electronic health record. And I had experience with electronic health records when I moonlit during my residency in urgent care at night and did a little bit of work away from UCLA for a year. And so I had the opportunity to see the power of information technology, and this was kind of at the beginning of the transition to electronic health records. So I got involved in that thinking it would just be a kind of short time thing. I'd be part of the implementation, but really fell in love with it and ended up becoming UCLA's first chief medical informatics officer, and the rest is history. So I've been very lucky to be able to combine clinical informatics and my love of caring for patients as a hospitalist.

Maya Adam:

Amazing. And can you explain in simple terms how digital health technologies like AI could really benefit patients? And maybe give us some examples.

Michael Pfeffer:

Absolutely. I like to think of it in two large buckets, augmentation and automation. And think about automation as AI is doing something that the human can do, and it allows us to basically take that task or burden away from the human so that the human can do something else. So for example, let's say that we could listen to a conversation between a patient and a clinician, and rather than the clinician having to go back and type a note or during the encounter type a note, we can write the note for them based on the conversation they had with the patient. That's always been kind of our dream and now it's reality. We actually have hundreds of physicians now live on that technology here at Stanford Healthcare, and it really does benefit the physician patient interaction Augmentation is when the human, in this case, the clinician and AI can do something that neither can do alone. So this is really making diagnoses, predicting disease, predicting who would benefit from a certain treatment and providing that information to the physician. I like to think of it, we've heard the term precision health. It's kind of enabling or can enable precision health.

Maya Adam:

And just to go back to that first example, if let's say an AI tool drafts notes, obviously the physician reviews those notes, right? It's not just sort of an automated get sent out without being checked.

Michael Pfeffer:

Correct. So it's all using human in the loop, that's what we call it, which is basically it is automating the task of someone typing the note, but requires the clinician to review the note, edit it, and sign it once it's complete. But what's really nice is you capture such rich detail from the patient in when we're getting the history of present illness, for example, that is just much better than trying to replicate that say later on in the day or in the evening, a lot of time when notes are written. So it's not only relieving some of the kind of time that it takes clinicians to write the notes, but also I believe will generate better notes, more accurate notes.

Maya Adam:

And Mike, I've met some patients who are understandably skeptical about the use of AI in their care. I'm curious about whether or not you think that's warranted.

Michael Pfeffer:

I think skepticism is warranted, and that's why we have an initiative here at Stanford Medicine called Raise Health, which is responsible AI for safe and equitable care. And we have a responsible AI life cycle at Stanford Healthcare. So we review all of the AI that we would be using for bias, for ethical concerns, for how it performs, for how we're going to continue to monitor it when it goes into production. We have a governance committee that reviews all of this, so we take it very, very seriously. I do believe though, that as this technology gets better and better and better, it's going to kind of be the reverse where you're going to want to see clinicians that use AI and not see clinicians that don't, because you're going to get better care. We're never going to get to perfect. I think if we aim for perfect, we're going to miss the opportunity to get better than where we are today. And so what's really exciting about it,

Maya Adam:

It reminds me of something I heard at Frontiers in Medicine that actually Jonathan Chen said at the end of his talk, he said he doesn't really see a risk of AI replacing physicians, but physicians who know how to use AI could replace physicians who don't. Do you agree with that?

Michael Pfeffer:

Yeah, I totally agree with that. And I think, again, beyond physicians, I mean, all clinicians have an opportunity to benefit from these technologies to reduce and eliminate tasks that really are not valuable per se, to the direct care of patients. And so there's a shortage of healthcare providers globally. So if we can use this technology to allow providers to see more patients more efficiently, that's a benefit for all. So it's not going to replace providers by any means. I still am waiting to see AI put in an IV into a patient. So I mean, the human aspect of medicine is so important, but giving more of that back to our clinicians, more of that time to do that to our clinicians is the power of ai.

Maya Adam:

And Mike, I'm curious about the process of training healthcare professionals on the use of AI in the clinic. Has it been challenging to get clinicians to buy into this?

Michael Pfeffer:

I think clinicians are very excited because again, we are trying to really give them the tools to take better care of patients, spend more time with patients, spend more time with their families. And so it's a big win all around, but it is not something that we're rolling out lightly. We have training. So for example, with the ambient AI-based scribe technology, the documentation that I talked about, we have training that goes along with that. So we're not going to just say, here, go use it. It's like, this is how you use it, here's the benefits. Here are the things you need to watch out for. And we'd like feedback as we continue down this process so we can continue to make it better. One of the things we've done at Stanford Medicine is we've actually have a generative AI tool that is safe for high risk data, basically.

And it allows our clinicians, staff, really everybody here to experiment with the capabilities of these tools. And perhaps the best way to learn what these tools can do is to actually use them. So yeah, I think there's tremendous potential as long as we're really focused, what are the problems that we're trying to solve? And can those problems be solved with ai? Not every problem can be solved with ai. Some problems don't need AI to be solved. The other really interesting piece about this in terms of AI and what it can do is what do you do with it? What do you do with the prediction? What do you do with the result? And that is just as important, if not more important when you think about what is the outcome of the AI itself. So you have to take into account both pieces. And so the AI in and of itself, we have to understand it limitations as well as the workflow that goes along with the AI to make sure it all comes together.

Maya Adam:

Mike, we are so lucky to have you. Thank you very much for making the time to chat with us today.

Michael Pfeffer:

It's a pleasure, Maya. And I really want to shout out the incredible team here at Technology and Digital Solutions that make all of this happen. Without an incredible team of technology professionals dedicated to our mission, this wouldn't happen. So I really want to recognize all the amazing work that has gone on and will go on to really improve our patient and clinician experience.

Maya Adam:

Excellent. Thank you, Mike. Thanks.