William Ratliff

William Ratliff

Auto-generated profile, based on publicly available information

William Ratliff

Duke Institute for Health Innovation, Durham, NC, United States

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 July 2020 in JMIR Medical Informatics
Verified
Journal Article
Open access
MSMark P Sendak
Mark P Sendak
WRWilliam Ratliff
William Ratliff
DSDina Sarro
Dina Sarro
EAElizabeth Alderton
Elizabeth Alderton
JFJoseph Futoma
Joseph Futoma
COCara O'Brien
Cara O'Brien
Background: Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature. Objective: This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care. Methods: In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch. Results: Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch. Conclusions: Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.

27 January 2020
Verified
Conference Paper
Open access
MSMark Sendak
Mark Sendak
MEMadeleine Clare Elish
Madeleine Clare Elish
MGMichael Gao
Michael Gao
JFJoseph Futoma
Joseph Futoma
WRWilliam Ratliff
William Ratliff
COCara O'Brien
Cara O'Brien

19 November 2020 in Journal of Medical Internet Research
Verified
Journal Article
Open access
SSSahil Sandhu
Sahil Sandhu
ALAnthony L Lin
Anthony L Lin
NBNathan Brajer
Nathan Brajer
JSJessica Sperling
Jessica Sperling
WRWilliam Ratliff
William Ratliff
MSMark P Sendak
Mark P Sendak
Background: Machine learning models have the potential to improve diagnostic accuracy and management of acute conditions. Despite growing efforts to evaluate and validate such models, little is known about how to best translate and implement these products as part of routine clinical care.Objective: This study aims to explore the factors influencing the integration of a machine learning sepsis early warning system (Sepsis Watch) into clinical workflows.Methods: We conducted semistructured interviews with 15 frontline emergency department physicians and rapid response team nurses who participated in the Sepsis Watch quality improvement initiative. Interviews were audio recorded and transcribed. We used a modified grounded theory approach to identify key themes and analyze qualitative data.Results: A total of 3 dominant themes emerged: perceived utility and trust, implementation of Sepsis Watch processes, and workforce considerations. Participants described their unfamiliarity with machine learning models. As a result, clinician trust was influenced by the perceived accuracy and utility of the model from personal program experience. Implementation of Sepsis Watch was facilitated by the easy-to-use tablet application and communication strategies that were developed by nurses to share model outputs with physicians. Barriers included the flow of information among clinicians and gaps in knowledge about the model itself and broader workflow processes.Conclusions: This study generated insights into how frontline clinicians perceived machine learning models and the barriers to integrating them into clinical workflows. These findings can inform future efforts to implement machine learning interventions in real-world settings and maximize the adoption of these interventions.

1 November 2020 in JCO Oncology Practice
Verified
Journal Article
Closed access
SSSahil Sandhu
Sahil Sandhu
ZKZoe King
Zoe King
MWMichelle Wong
Michelle Wong
SBSean Bissell
Sean Bissell
JSJessica Sperling
Jessica Sperling
TLThomas W. LeBlanc
Thomas W. LeBlanc
PURPOSE: Electronic patient-reported outcomes (ePROs) can help clinicians proactively assess and manage their patients’ symptoms. Despite known benefits, there is limited adoption of ePROs into routine clinical care as a result of workflow and technologic challenges. This study identifies oncologists’ perspectives on factors that affect integration of ePROs into clinical workflows. METHODS: We conducted semistructured qualitative interviews with 16 oncologists from a large academic medical center, across diverse subspecialties and cancer types. Oncologists were asked how they currently use or could imagine using ePROs before, during, and after a patient visit. We used an inductive approach to thematically analyze these qualitative data. RESULTS: Results were categorized into the following three main themes: (1) selection and development of ePRO tool, (2) contextual drivers of adoption, and (3) patient-facing concerns. Respondents preferred diagnosis-based ePRO tools over more general symptom screeners. Although they noted information overload as a potential barrier, respondents described strong data visualization and ease of use as facilitators. Contextual drivers of oncologist adoption include identifying target early adopters, incentivizing uptake through use of ePRO data to support billing and documentation, and emphasizing benefits for patient care and efficiency. Respondents also indicated the need to focus on patient-facing issues, such as patient response rate, timing of survey distribution, and validity and reliability of responses. DISCUSSION: Respondents identified several barriers and facilitators to successful uptake of ePROs. Understanding oncologists’ perspectives is essential to inform both practice-level implementation strategies and policy-level decisions to include ePROs in alternative payment models for cancer care.

9 April 2024 in npj Digital Medicine
Verified
Journal Article
Open access
PDF file
WBWilliam Boag
William Boag
AHAlifia Hasan
Alifia Hasan
JKJee Young Kim
Jee Young Kim
MRMike Revoir
Mike Revoir
MNMarshall Nichols
Marshall Nichols
MSMark Sendak
Mark Sendak
When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This “Algorithm Journey Map” enumerates all social and technical activities throughout the AI solution’s procurement, development, integration, and full lifecycle management. In addition to mapping the “who?” and “what?” of the adoption of the AI tool, we also show several ‘lessons learned’ throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles—in algorithmic systems.

20 September 2023 in JMIR Research Protocols
Verified
Journal Article
Open access
DMDavid Y Ming
David Y Ming
WWWillis Wong
Willis Wong
KJKelley A Jones
Kelley A Jones
RARichard C Antonelli
Richard C Antonelli
NGNitin Gujral
Nitin Gujral
HKHeather A King
Heather A King
Background: Electronic health record (EHR)–integrated digital personal health records (PHRs) via Fast Healthcare Interoperability Resources (FHIR) are promising digital health tools to support care coordination (CC) for children and youth with special health care needs but remain widely unadopted; as their adoption grows, mixed methods and implementation research could guide real-world implementation and evaluation. Objective: This study (1) evaluates the feasibility of an FHIR-enabled digital PHR app for CC for children and youth with special health care needs, (2) characterizes determinants of implementation, and (3) explores associations between adoption and patient- or family-reported outcomes. Methods: This nonrandomized, single-arm, prospective feasibility trial will test an FHIR-enabled digital PHR app’s use among families of children and youth with special health care needs in primary care settings. Key app features are FHIR-enabled access to structured data from the child’s medical record, families’ abilities to longitudinally track patient- or family-centered care goals, and sharing progress toward care goals with the child’s primary care provider via a clinician dashboard. We shall enroll 40 parents or caregivers of children and youth with special health care needs to use the app for 6 months. Inclusion criteria for children and youth with special health care needs are age 0-16 years; primary care at a participating site; complex needs benefiting from CC; high hospitalization risk in the next 6 months; English speaking; having requisite technology at home (internet access, Apple iOS mobile device); and an active web-based EHR patient portal account to which a parent or caregiver has full proxy access. Digital prescriptions will be used to disseminate study recruitment materials directly to eligible participants via their existing EHR patient portal accounts. We will apply an intervention mixed methods design to link quantitative and qualitative (semistructured interviews and family engagement panels with parents of children and youth with special health care needs) data and characterize implementation determinants. Two CC frameworks (Pediatric Care Coordination Framework; Patient-Centered Medical Home) and 2 evaluation frameworks (Consolidated Framework for Implementation Research; Technology Acceptance Model) provide theoretical foundations for this study. Results: Participant recruitment began in fall 2022, before which we identified >300 potentially eligible patients in EHR data. A family engagement panel in fall 2021 generated formative feedback from family partners. Integrated analysis of pretrial quantitative and qualitative data informed family-centered enhancements to study procedures. Conclusions: Our findings will inform how to integrate an FHIR-enabled digital PHR app for children and youth with special health care needs into clinical care. Mixed methods and implementation research will help strengthen implementation in diverse clinical settings. The study is positioned to advance knowledge of how to use digital health innovations for improving care and outcomes for children and youth with special health care needs and their families. Trial Registration: ClinicalTrials.gov NCT05513235; https://clinicaltrials.gov/study/NCT05513235 International Registered Report Identifier (IRRID): DERR1-10.2196/46847