Armando Bedoya

Armando Bedoya


Research Focus & Metrics

Academic Career Length

17Years (2010-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 July 2020 in JMIR Medical Informatics
Verified
Journal Article
Open access
MSMark P Sendak
Mark P Sendak
WRWilliam Ratliff
William Ratliff
DSDina Sarro
Dina Sarro
EAElizabeth Alderton
Elizabeth Alderton
JFJoseph Futoma
Joseph Futoma
COCara O'Brien
Cara O'Brien
Background: Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature. Objective: This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care. Methods: In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch. Results: Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch. Conclusions: Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.

19 November 2020 in Journal of Medical Internet Research
Verified
Journal Article
Open access
SSSahil Sandhu
Sahil Sandhu
ALAnthony L Lin
Anthony L Lin
NBNathan Brajer
Nathan Brajer
JSJessica Sperling
Jessica Sperling
WRWilliam Ratliff
William Ratliff
MSMark P Sendak
Mark P Sendak
Background: Machine learning models have the potential to improve diagnostic accuracy and management of acute conditions. Despite growing efforts to evaluate and validate such models, little is known about how to best translate and implement these products as part of routine clinical care.Objective: This study aims to explore the factors influencing the integration of a machine learning sepsis early warning system (Sepsis Watch) into clinical workflows.Methods: We conducted semistructured interviews with 15 frontline emergency department physicians and rapid response team nurses who participated in the Sepsis Watch quality improvement initiative. Interviews were audio recorded and transcribed. We used a modified grounded theory approach to identify key themes and analyze qualitative data.Results: A total of 3 dominant themes emerged: perceived utility and trust, implementation of Sepsis Watch processes, and workforce considerations. Participants described their unfamiliarity with machine learning models. As a result, clinician trust was influenced by the perceived accuracy and utility of the model from personal program experience. Implementation of Sepsis Watch was facilitated by the easy-to-use tablet application and communication strategies that were developed by nurses to share model outputs with physicians. Barriers included the flow of information among clinicians and gaps in knowledge about the model itself and broader workflow processes.Conclusions: This study generated insights into how frontline clinicians perceived machine learning models and the barriers to integrating them into clinical workflows. These findings can inform future efforts to implement machine learning interventions in real-world settings and maximize the adoption of these interventions.

31 May 2022 in Journal of the American Medical Informatics Association
Verified
Journal Article
Closed access
PDF file
ABArmando Bedoya
Armando Bedoya
NENicoleta J Economou-Zavlanos
Nicoleta J Economou-Zavlanos
BGBenjamin A Goldstein
Benjamin A Goldstein
AYAllison Young
Allison Young
JJJ Eric Jelovsek
J Eric Jelovsek
MPMichael J Pencina
Michael J Pencina
Artificial intelligence/machine learning models are being rapidly developed and used in clinical practice. However, many models are deployed without a clear understanding of clinical or operational impact and frequently lack monitoring plans that can detect potential safety signals. There is a lack of consensus in establishing governance to deploy, pilot, and monitor algorithms within operational healthcare delivery workflows. Here, we describe a governance framework that combines current regulatory best practices and lifecycle management of predictive models being used for clinical care. Since January 2021, we have successfully added models to our governance portfolio and are currently managing 52 models.

11 April 2020 in JAMIA Open
Verified
Journal Article
Open access
PDF file
ABArmando Bedoya
Armando Bedoya
JFJoseph Futoma
Joseph Futoma
Meredith ClementMeredith Clement
Meredith Clement
KCKristin Corey
Kristin Corey
NBNathan Brajer
Nathan Brajer
COCara O’brien
Cara O’brien
Objective: Determine if deep learning detects sepsis earlier and more accurately than other models. To evaluate model performance using implementation-oriented metrics that simulate clinical practice.Materials and Methods: We trained internally and temporally validated a deep learning model (multi-output Gaussian process and recurrent neural network [MGP–RNN]) to detect sepsis using encounters from adult hospitalized patients at a large tertiary academic center. Sepsis was defined as the presence of 2 or more systemic inflammatory response syndrome (SIRS) criteria, a blood culture order, and at least one element of end-organ failure. The training dataset included demographics, comorbidities, vital signs, medication administrations, and labs from October 1, 2014 to December 1, 2015, while the temporal validation dataset was from March 1, 2018 to August 31, 2018. Comparisons were made to 3 machine learning methods, random forest (RF), Cox regression (CR), and penalized logistic regression (PLR), and 3 clinical scores used to detect sepsis, SIRS, quick Sequential Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS). Traditional discrimination statistics such as the C-statistic as well as metrics aligned with operational implementation were assessed.Results: The training set and internal validation included 42 979 encounters, while the temporal validation set included 39 786 encounters. The C-statistic for predicting sepsis within 4 h of onset was 0.88 for the MGP–RNN compared to 0.836 for RF, 0.849 for CR, 0.822 for PLR, 0.756 for SIRS, 0.619 for NEWS, and 0.481 for qSOFA. MGP–RNN detected sepsis a median of 5 h in advance. Temporal validation assessment continued to show the MGP–RNN outperform all 7 clinical risk score and machine learning comparisons.Conclusions: We developed and validated a novel deep learning model to detect sepsis. Using our data elements and feature set, our modeling approach outperformed other machine learning methods and clinical scores.

1 April 2014 in American Journal of Transplantation
Verified
Journal Article
Open access
PDF file
LSL. D. Snyder
L. D. Snyder
AGA. L. Gray
A. L. Gray
JRJ. M. Reynolds
J. M. Reynolds
GAG. M. Arepally
G. M. Arepally
ABArmando Bedoya
Armando Bedoya
SPS. M. Palmer
S. M. Palmer
As HLAs antibody detection technology has evolved, there is now detailed HLA antibody information available on prospective transplant recipients. Determining single antigen antibody specificity allows for a calculated panel reactive antibodies (cPRA) value, providing an estimate of the effective donor pool. For broadly sensitized lung transplant candidates (cPRA ≥ 80%), our center adopted a pretransplant multi-modal desensitization protocol in an effort to decrease the cPRA and expand the donor pool. This desensitization protocol included plasmapheresis, solumedrol, bortezomib and rituximab given in combination over 19 days followed by intravenous immunoglobulin. Eight of 18 candidates completed therapy with the primary reasons for early discontinuation being transplant (by avoiding unacceptable antigens) or thrombocytopenia. In a mixed-model analysis, there were no significant changes in PRA or cPRA changes over time with the protocol. A sub-analysis of the median fluorescence intensity (MFI) change indicated a small decline that was significant in antibodies with MFI 5000-10,000. Nine of 18 candidates subsequently had a transplant. Posttransplant survival in these nine recipients was comparable to other pretransplant-sensitized recipients who did not receive therapy. In summary, an aggressive multi-modal desensitization protocol does not significantly reduce pretransplant HLA antibodies in a broadly sensitized lung transplant candidate cohort.

29 November 2023 in Journal of the American Medical Informatics Association
Verified
Journal Article
Closed access
PDF file
NENicoleta Economou
Nicoleta Economou
SBSophia Bessias
Sophia Bessias
MCMichael P Cary
Michael P Cary
ABArmando Bedoya
Armando Bedoya
BGBenjamin A Goldstein
Benjamin A Goldstein
EPEric G Poon
Eric G Poon
Objective: The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution’s approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. Materials and Methods: Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. Results: An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. Discussion: By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. Conclusions: We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.