Rachel Kim

Rachel Kim

Auto-generated profile, based on publicly available information

Rachel Kim

Colorado Center for Personalized Medicine University of Colorado Anschutz Medical Campus Aurora, CO United States

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 August 2022 in JMIR Formative Research
Verified
Journal Article
Open access
JSJohn Stacy
John Stacy
RKRachel Kim
Rachel Kim
CBChristopher Barrett
Christopher Barrett
BSBalaviknesh Sekar
Balaviknesh Sekar
SSSteven Simon
Steven Simon
MRMichael A Rosenberg
Michael A Rosenberg
Background: Despite the numerous studies evaluating various rhythm control strategies for atrial fibrillation (AF), determination of the optimal strategy in a single patient is often based on trial and error, with no one-size-fits-all approach based on international guidelines/recommendations. The decision, therefore, remains personal and lends itself well to help from a clinical decision support system, specifically one guided by artificial intelligence (AI). QRhythm utilizes a 2-stage machine learning (ML) model to identify the optimal rhythm management strategy in a given patient based on a set of clinical factors, in which the model first uses supervised learning to predict the actions of an expert clinician and identifies the best strategy through reinforcement learning to obtain the best clinical outcome—a composite of symptomatic recurrence, hospitalization, and stroke. Objective: We qualitatively evaluated a novel, AI-based, clinical decision support system (CDSS) for AF rhythm management, called QRhythm, which uses both supervised and reinforcement learning to recommend either a rate control or one of 3 types of rhythm control strategies—external cardioversion, antiarrhythmic medication, or ablation—based on individual patient characteristics. Methods: Thirty-three clinicians, including cardiology attendings and fellows and internal medicine attendings and residents, performed an assessment of QRhythm, followed by a survey to assess relative comfort with automated CDSS in rhythm management and to examine areas for future development. Results: The 33 providers were surveyed with training levels ranging from resident to fellow to attending. Of the characteristics of the app surveyed, safety was most important to providers, with an average importance rating of 4.7 out of 5 (SD 0.72). This priority was followed by clinical integrity (a desire for the advice provided to make clinical sense; importance rating 4.5, SD 0.9), backward interpretability (transparency in the population used to create the algorithm; importance rating 4.3, SD 0.65), transparency of the algorithm (reasoning underlying the decisions made; importance rating 4.3, SD 0.88), and provider autonomy (the ability to challenge the decisions made by the model; importance rating 3.85, SD 0.83). Providers who used the app ranked the integrity of recommendations as their highest concern with ongoing clinical use of the model, followed by efficacy of the application and patient data security. Trust in the app varied; 1 (17%) provider responded that they somewhat disagreed with the statement, “I trust the recommendations provided by the QRhythm app,” 2 (33%) providers responded with neutrality to the statement, and 3 (50%) somewhat agreed with the statement. Conclusions: Safety of ML applications was the highest priority of the providers surveyed, and trust of such models remains varied. Widespread clinical acceptance of ML in health care is dependent on how much providers trust the algorithms. Building this trust involves ensuring transparency and interpretability of the model.

6 December 2021 in JMIR Medical Informatics
Verified
Journal Article
Open access
RKRachel S Kim
Rachel S Kim
SSSteven Simon
Steven Simon
BPBrett Powers
Brett Powers
ASAmneet Sandhu
Amneet Sandhu
JSJose Sanchez
Jose Sanchez
MRMichael A Rosenberg
Michael A Rosenberg
Background: The identification of an appropriate rhythm management strategy for patients diagnosed with atrial fibrillation (AF) remains a major challenge for providers. Although clinical trials have identified subgroups of patients in whom a rate- or rhythm-control strategy might be indicated to improve outcomes, the wide range of presentations and risk factors among patients presenting with AF makes such approaches challenging. The strength of electronic health records is the ability to build in logic to guide management decisions, such that the system can automatically identify patients in whom a rhythm-control strategy is more likely and can promote efficient referrals to specialists. However, like any clinical decision support tool, there is a balance between interpretability and accurate prediction. Objective: This study aims to create an electronic health record–based prediction tool to guide patient referral to specialists for rhythm-control management by comparing different machine learning algorithms. Methods: We compared machine learning models of increasing complexity and used up to 50,845 variables to predict the rhythm-control strategy in 42,022 patients within the University of Colorado Health system at the time of AF diagnosis. Models were evaluated on the basis of their classification accuracy, defined by the F1 score and other metrics, and interpretability, captured by inspection of the relative importance of each predictor. Results: We found that age was by far the strongest single predictor of a rhythm-control strategy but that greater accuracy could be achieved with more complex models incorporating neural networks and more predictors for each participant. We determined that the impact of better prediction models was notable primarily in the rate of inappropriate referrals for rhythm-control, in which more complex models provided an average of 20% fewer inappropriate referrals than simpler, more interpretable models. Conclusions: We conclude that any health care system seeking to incorporate algorithms to guide rhythm management for patients with AF will need to address this trade-off between prediction accuracy and model interpretability.

14 January 2022 in JMIR Preprints
Verified
Preprint
Open access
JSJohn Stacy
John Stacy
RKRachel Kim
Rachel Kim
CBChristopher D Barrett
Christopher D Barrett
BSBalaviknesh Sekar
Balaviknesh Sekar
SSSteve Simon
Steve Simon
MRMichael A. Rosenberg
Michael A. Rosenberg
BACKGROUND Despite the numerous studies evaluating various rhythm-control strategies for patients with atrial fibrillation, determination of the optimal strategy in a single patient is often based on trial-and-error, with no one-size-fits-all approach based on international guideline recommendations. The decision, therefore, remains an individualized one that lends itself well to the aid of a clinical decision support system, specifically one guided by artificial intelligence (AI). OBJECTIVE In this study, we performed qualitative evaluation of a novel, AI-based, clinical decision support system (CDSS) for rhythm management of AF called QRhythm, which uses a combination of supervised and reinforcement learning to recommend either a rate-control or one of three types of rhythm-control strategies—external cardioversion, antiarrhythmic medication, or ablation—based on individual patient characteristics. METHODS QRhythm utilizes a two-stage machine learning model to identify the optimal rhythm-management strategy in a given patient based on a set of clinical factors, in which the model first uses supervised learning to mimic an expert clinician, and then applies reinforcement learning to identify the best strategy to obtain the best clinical outcome—a composite of symptomatic recurrence, hospitalization, and stroke. 33 clinical providers, including cardiologists and internal medicine physicians, performed an assessment of QRhythm, followed by a survey to assess relative comfort with automated CDSS in rhythm-management, and examine areas for future development. RESULTS 33 providers were surveyed with training levels ranging from resident to fellow to attending. Of the characteristics of the app surveyed, safety was most important to providers with an average importance rating of 4.7 out of 5 (SD 0.72). This priority was followed by a desire for the advice provided to make clinical sense with an importance of 4.5 out of 5 (SD 0.9), transparency in the population used to create the algorithm, 4.3 out of 5 (SD 0.65), and reasoning behind the decisions made, 4.3 out of 5 (SD0.88), and the ability to challenge the decisions made by the model, 3.85 out of 5 (SD 0.83). Providers that used the app ranked their highest concern with ongoing clinical use of the model as accuracy of recommendations followed by ineffectiveness of the application and patient data security. Trust in the app was varied. 16.7% of providers responded that they somewhat disagreed with the statement “I trust the recommendations provided by the QRhythm app,” 33.3% responded with neutrality to the statement, and 50% somewhat agreed with the statement. CONCLUSIONS Safety of machine learning applications was the number one priority of the providers surveyed, and trust of such models remains varied. Widespread clinical acceptance of machine learning in health care is dependent on how much providers trust the algorithms. Building this trust involves ensuring transparency and interpretability of the model.

21 March 2022 in JMIR Preprints
Verified
Preprint
Open access
CBChristopher D Barrett
Christopher D Barrett
RKRachel Kim
Rachel Kim
ESEric Sink
Eric Sink
ATAlexis Tumolo
Alexis Tumolo
MZMatthew Zipse
Matthew Zipse
MRMichael A. Rosenberg
Michael A. Rosenberg
BACKGROUND Decisions regarding initial rhythm management strategy for patients with newly diagnosed atrial fibrillation (AF) are challenging to individualize. Prior work by our team has demonstrated feasibility of using machine learning algorithms at the level of diagnostic codes to predict rhythm management strategies; however, this work did not examine prediction models at the individual level. OBJECTIVE We aimed to examine decisions about rhythm management for AF and associated outcomes at the individual patient level, through mapping clinical trajectories within the electronic health record. We examined use of quantitative methods to predict the use of rate- or rhythm-control therapy in patients with a new diagnosis of AF. METHODS We performed a targeted chart review of patients with a new diagnosis of atrial fibrillation in the University of Colorado healthcare system from 2011 to 2020. We used clinical and demographic information to develop and test machine learning algorithms for predicting initial rhythm management strategy, and compared models based on accuracy as well as interpretability. Time-to-event and regression analyses were performed to predict the likelihood of change in rhythm management and the risk of subsequent hospitalization or death. RESULTS Of 419 patients with an EHR-diagnosis of new AF, we confirmed the diagnosis in 289 patients, 194 of whom we classified as paroxysmal and 95 persistent. For all patients with AF, rhythm-control therapy resulted in more total management changes. For paroxysmal AF, rhythm-control therapy resulted in fewer hospitalizations (incidence rate ratio 0.32, 95% CI 0.19-0.34) and no difference in mortality compared to rate-control therapy. For persistent AF, rhythm control resulted in more hospitalizations (incidence rate ratio 2.44, 95% CI 1.41-4.22) with no difference in mortality. Machine learning models could predict the initial strategy with limited accuracy, which was higher in patients with persistent than paroxysmal AF. CONCLUSIONS Quantitative decision models for rhythm management of AF can be developed through trajectory mapping using chart review. Discussion about patient-centered outcomes should be considered when selecting initial therapy for atrial fibrillation. CLINICALTRIAL n/a

30 March 2021 in JMIR Preprints
Verified
Preprint
Open access
RKRachel S Kim
Rachel S Kim
SSSteven Simon
Steven Simon
BPBrett Powers
Brett Powers
ASAmneet Sandhu
Amneet Sandhu
JSJose Sanchez
Jose Sanchez
MRMichael A Rosenberg
Michael A Rosenberg
BACKGROUND The identification of an appropriate rhythm management strategy for patients diagnosed with atrial fibrillation (AF) remains a major challenge for providers. Although clinical trials have identified subgroups of patients in whom a rate- or rhythm-control strategy might be indicated to improve outcomes, the wide range of presentations and risk factors among patients presenting with AF makes such approaches challenging. The strength of electronic health records is the ability to build in logic to guide management decisions, such that the system can automatically identify patients in whom a rhythm-control strategy is more likely and can promote efficient referrals to specialists. However, like any clinical decision support tool, there is a balance between interpretability and accurate prediction. OBJECTIVE This study aims to create an electronic health record–based prediction tool to guide patient referral to specialists for rhythm-control management by comparing different machine learning algorithms. METHODS We compared machine learning models of increasing complexity and used up to 50,845 variables to predict the rhythm-control strategy in 42,022 patients within the University of Colorado Health system at the time of AF diagnosis. Models were evaluated on the basis of their classification accuracy, defined by the F1 score and other metrics, and interpretability, captured by inspection of the relative importance of each predictor. RESULTS We found that age was by far the strongest single predictor of a rhythm-control strategy but that greater accuracy could be achieved with more complex models incorporating neural networks and more predictors for each participant. We determined that the impact of better prediction models was notable primarily in the rate of inappropriate referrals for rhythm-control, in which more complex models provided an average of 20% fewer inappropriate referrals than simpler, more interpretable models. CONCLUSIONS We conclude that any health care system seeking to incorporate algorithms to guide rhythm management for patients with AF will need to address this trade-off between prediction accuracy and model interpretability.