Who's Responsible When AI Gets It Wrong?

Most conversations about AI in healthcare focus on what the technology can do. This one is about what it should do, and who gets to decide.

In the latest episode of MDAware: Upgrading Clinical Judgment, host Dr. Graham Walker sits down with Dr. Sarah Gebauer, a physician and healthcare AI governance expert whose career has spanned anesthesiology, hospice and palliative care, consulting, and national security AI evaluation for RAND. The two have known each other since medical school, and their conversation has the kind of candor that comes with that history.

Meet the Guest

Dr. Sarah Gebauer is a practicing rural anesthesiologist and one of the leading physician voices in healthcare AI governance. She has held leadership positions across multiple health systems, contributed to AI evaluation work at RAND, and was one of the key contributors to the MDCalc Physician's Charter — a set of values and practical guidelines for AI in medicine, published in August 2023.

Where the Physician's Charter Came From

The Charter started with a LinkedIn post and a question MDCalc co-founder Dr. Joe Habboushe couldn't shake: what does the future hold if this technology goes sideways? Rather than wait for someone else to answer it, the MDCalc team opened the question up to a broad group of practicing physicians and built something from the ground up; ten rules of the road, grounded in the four pillars of medical ethics, with chapters written by contributors across specialties.

Three years later, Walker and Gebauer use this episode to audit that document honestly. Some of it holds up well. Some of it didn't anticipate how fast things would move.

The Issues the Field Still Hasn't Resolved

The conversation gets specific in ways that most AI discussions don't. Gebauer walks through why HIPAA's definition of de-identified data may no longer be adequate - AI is remarkably good at re-identifying patients from data that looks anonymous on the surface. She also unpacks the tension between building accurate, unbiased AI models and protecting the privacy of the patients whose data makes those models possible. These aren't hypothetical problems. They're happening now, mostly out of public view.

Walker adds another angle that doesn't get enough attention: sycophancy. Most large language models are trained to be agreeable. That might be fine for a productivity tool. It's a more complicated trait in a system that's helping a clinician or a patient make a high-stakes decision.

What a 2026 Update Needs to Address

The original Charter was written when generative AI was still new enough to feel magical. The 2026 version, which Walker and Gebauer agree needs to be written, would have to grapple with a different landscape entirely- AI agents operating in chains, foundation models that make questions of data provenance harder to answer, and patient-facing tools that increasingly cut the physician out of the loop altogether.

Worth Knowing

If you work in a clinical setting, use AI tools in your practice, or are thinking about how your institution should be approaching any of this, this episode is worth your time. It is not a breathless endorsement of AI in medicine, and it is not a warning against it. It is a clear-eyed look at where things actually stand.

MDAware: Upgrading Clinical Judgment is MDCalc's podcast dedicated to breaking down the science behind clinical scores and bringing transparency and confidence to your clinical workflow.