Anne Gravel Sullivan

Anne Gravel Sullivan

Auto-generated profile, based on publicly available information

Anne Gravel Sullivan

Institute for Clinical and Translational Research, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 November 2025 in NEJM AI
Verified
Journal Article
Closed access
MAMajid Afshar
Majid Afshar
MBMary Ryan Baumann
Mary Ryan Baumann
FRFelice Resnik
Felice Resnik
JHJosie Hintzke
Josie Hintzke
ASAnne Gravel Sullivan
Anne Gravel Sullivan
JGJoel Gordon
Joel Gordon
Electronic health record (EHR) documentation is a major contributor to work-related practitioner exhaustion and the interpersonal disengagement known as burnout. Generative artificial intelligence (AI) scribes that passively capture clinical conversations and draft visit notes may alleviate this burden, but evidence remains limited. A 24-week, stepped-wedge, individually randomized pragmatic trial was conducted across ambulatory clinics in two states. Sixty-six health care practitioners were randomly assigned to three 6-week sequences of ambient AI. The coprimary outcomes were professional fulfillment and work exhaustion/interpersonal disengagement from the Stanford Professional Fulfillment Index. Secondary measures included time spent on notes, work outside work (WoW), documentation quality with the Provider Documentation Summarization Quality Instrument 9 (PDSQI-9), and billing diagnostic codes reviewed by professional staff coders. Linear mixed models were used for intention-to-treat (ITT) analyses. A total of 71,487 notes were authored, of which 27,092 (38%) were generated using ambient AI. Ambient AI use had a significant reduction in work exhaustion/interpersonal disengagement (−0.44 points; 95% confidence interval [CI], −0.62 to −0.25; P<0.001), and a nonsignificant increase in professional fulfillment (+0.14 points; 95% CI, 0.004 to 0.28; P=0.04) on a five-point Likert scale. Among secondary measures, time spent on notes decreased (−0.36 hours per day; 95% CI, −0.55 to −0.17). The reduction in WoW (−0.50 hours per day; 95% CI, −0.90 to −0.09) was sensitive to exclusion of extreme values and was no longer significant after removing the top 3% of daily observations. Diagnostic billing codes improved with ambient AI use (P<0.001). Documentation quality, assessed with the PDSQI-9, demonstrated mean scores ranging from 3.97 to 4.99 across domains on a five-point scale. No drift in software performance was detected. In a real-world randomized implementation, ambient AI reduced health care practitioners’ work exhaustion/interpersonal disengagement but did not significantly increase professional fulfillment. Documentation time decreased without compromising diagnosis, billing compliance, or note quality. (Funded by the University of Wisconsin Hospital and Clinics and the National Institutes of Health Clinical and Translational Science Award; ClinicalTrials.gov number, NCT06517082.)

28 August 2025 in NEJM AI
Verified
Journal Article
Closed access
MAMajid Afshar
Majid Afshar
FRFelice Resnik
Felice Resnik
MBMary Ryan Baumann
Mary Ryan Baumann
JHJosie Hintzke
Josie Hintzke
KLKayla Lemmon
Kayla Lemmon
JGJoel E. Gordon
Joel E. Gordon
Ambient artificial intelligence (AI) offers the potential to reduce documentation burden and improve efficiency through clinical note generation. Widespread adoption, however, remains limited due to challenges in electronic health record (EHR) integration, coding compliance, and real-world evaluation. This study introduces a framework and protocols to design, monitor, and deploy ambient AI within routine care. We launched an implementation phase to build technical workflows, establish governance, and inform a pragmatic randomized trial. A bidirectional governance model linked operations and research through multidisciplinary workgroups that incorporated the Systems Engineering Initiative for Patient Safety (SEIPS) framework. Integration into the EHR used Fast Healthcare Interoperability Resources (FHIR), and a real-time dashboard tracked utilization and documentation accuracy. To monitor drift, a difference-in-differences analysis was applied to three process metrics: time in notes, work outside work, and utilization. Audits of International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) compliance were performed using an internally developed large language model (LLM), the validity of which was assessed through correlation with certified professional coders. Ambient AI utilization, measured as the proportion of eligible clinical notes completed using the system, had a weighted median of 65.4% (interquartile range, 50.6 to 84.0%). Iterative improvement cycles targeted task-specific adoption. A brief workflow issue related to a note template change initially reduced ICD-10 documentation accuracy from 79% (95% confidence interval [CI], 72 to 86%) to 35% (95% CI, 28 to 42%); accuracy returned to baseline after note template redesign and user training. The internally developed LLM coder achieved a strong correlation with professional coders (Pearson’s r=0.97). The trial enrolled 66 providers across eight specialties, powered at 90% for the primary outcome of provider well-being. We provide a publicly available framework and protocols to help safely implement ambient AI in health care. Innovations include an embedded pragmatic trial design, human factors engineering, compliance-driven feedback loops, and real-time monitoring to support deployment, ensuring fidelity before initiation of the clinical trial. (Funded by the University of Wisconsin Hospital and Clinics and the National Institutes of Health Clinical and Translational Science Award; NIH/ NCATS UL1TR002737; ClinicalTrials.gov number, NCT06517082.)