Hyungjin Tom Kim

Hyungjin Tom Kim

Auto-generated profile, based on publicly available information

Hyungjin Tom Kim

Department of Psychiatry and Human Behavior, Alpert Medical School, Brown University, Providence, RI, United States

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 November 2025 in JMIR AI
Verified
Journal Article
Open access
YHYining Hua
Yining Hua
WXWinna Xia
Winna Xia
DBDavid Bates
David Bates
GHGeorge Luke Hartstein
George Luke Hartstein
HKHyungjin Tom Kim
Hyungjin Tom Kim
JTJohn Torous
John Torous
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Health care chatbots are rapidly proliferating, while generative artificial intelligence (AI) outpaces existing evaluation standards. Objective: We aimed to develop a structured, stakeholder-informed framework to standardize evaluation of health care chatbots. Methods: PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses)–guided searches across multiple databases identified 266 records; 152 were screened, 21 full texts were assessed, and 11 frameworks were included. We extracted 356 questions (refined to 271 by deduplication and relevance review), mapped items to Coalition for Health AI constructs, and organized them with iterative input from clinicians, patients, developers, epidemiologists, and policymakers. Results: We developed the Health Care AI Chatbot Evaluation Framework (HAICEF), a hierarchical framework with 3 priority domains (safety, privacy, and fairness; trustworthiness and usefulness; and design and operational effectiveness) and 18 second-level and 60 third-level constructs covering 271 questions. Emphasis includes data provenance and harm control; Health Insurance Portability and Accountability Act/General Data Protection Regulation–aligned privacy and security; bias management; and reliability, transparency, and workflow integration. Question distribution across domains is as follows: design and operational effectiveness, 40%; trustworthiness and usefulness, 39%; and safety, privacy and fairness, 21%. The framework accommodates both patient-facing and back-office use cases. Conclusions: HAICEF provides an adaptable scaffold for standardized evaluation and responsible implementation of health care chatbots. Planned next steps include prospective validation across settings and a Delphi consensus to extend accountability and accessibility assurances.

19 November 2024 in JMIR Preprints
Verified
Preprint
Open access
YHYining Hua
Yining Hua
WXWinna Xia
Winna Xia
DBDavid Bates
David Bates
GHGeorge Luke Hartstein
George Luke Hartstein
HKHyungjin Tom Kim
Hyungjin Tom Kim
JTJohn Torous
John Torous
BACKGROUND Health care chatbots are rapidly proliferating, while generative artificial intelligence (AI) outpaces existing evaluation standards. OBJECTIVE We aimed to develop a structured, stakeholder-informed framework to standardize evaluation of health care chatbots. METHODS PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses)–guided searches across multiple databases identified 266 records; 152 were screened, 21 full texts were assessed, and 11 frameworks were included. We extracted 356 questions (refined to 271 by deduplication and relevance review), mapped items to Coalition for Health AI constructs, and organized them with iterative input from clinicians, patients, developers, epidemiologists, and policymakers. RESULTS We developed the Health Care AI Chatbot Evaluation Framework (HAICEF), a hierarchical framework with 3 priority domains (safety, privacy, and fairness; trustworthiness and usefulness; and design and operational effectiveness) and 18 second-level and 60 third-level constructs covering 271 questions. Emphasis includes data provenance and harm control; Health Insurance Portability and Accountability Act/General Data Protection Regulation–aligned privacy and security; bias management; and reliability, transparency, and workflow integration. Question distribution across domains is as follows: design and operational effectiveness, 40%; trustworthiness and usefulness, 39%; and safety, privacy and fairness, 21%. The framework accommodates both patient-facing and back-office use cases. CONCLUSIONS HAICEF provides an adaptable scaffold for standardized evaluation and responsible implementation of health care chatbots. Planned next steps include prospective validation across settings and a Delphi consensus to extend accountability and accessibility assurances.

21 May 2026 in JMIR Medical Education
Verified
Journal Article
Open access
JKJoshua Kemp
Joshua Kemp
ARAriella Rosen
Ariella Rosen
HKHyungjin Kim
Hyungjin Kim
MLMargo Adams Larsen
Margo Adams Larsen
KBKristen Benito
Kristen Benito
PTPeter Tuerk
Peter Tuerk
Background: Exposure-based cognitive behavioral therapy is among the least used evidence-based practices for anxiety disorders in routine care. Providers’ negative beliefs about exposure (eg, fears of harm or intolerability) are a major barrier. Experiential methods can reduce these beliefs but are limited by accessibility, standardization, and fidelity. Virtual reality (VR) offers a scalable way to deliver standardized experiential practice. Guided by an “exposure to exposure” (E2E) framework, we conceptualized VR training as an exposure intervention targeting therapists’ own anxious beliefs about exposure. Objective: This feasibility study examined a VR-based exposure training program (SET-VR ) (1) to evaluate usability and effects on therapist learning targets (knowledge, self-efficacy, attitudes) and (2) to test whether a high-immersion head-mounted display (HMD) format provides added benefit over a lower-immersion desktop format. Eligibility included holding an active caseload. Methods: Eligible clinicians (ie, aged >21 years with an active caseload; n=41) completed a 4-hour didactic workshop on exposure and were randomized (1:1, blinded) to the desktop or HMD condition. In the experiential phase, therapists delivered 3 rounds of exposure with a virtual patient. They titrated exposure intensity (increase, decrease, continue as is) at fixed decision points based on state-dependent visual (character animations) and auditory (prerecorded verbalizations) cues reflecting the patient’s distress. Exposure knowledge, self-efficacy, and beliefs about exposure were measured at baseline, post-didactic, post-experiential, and follow-up. Participants also rated the acceptability, usability, and authenticity of the program. Results: Both groups (desktop and HMD) showed significant improvement in exposure knowledge ( d =0.52, P =.006; d =0.58, P =.002), self-efficacy ( d =0.88, P <.001; d =1.36, P <.001), and beliefs ( d =0.61, P =.001; d =1.05, P <.001) from baseline to post-didactic training using binomial generalized estimating equations. There were no significant differences between the low- and high-immersion groups on any measure after didactics. Both groups demonstrated significant improvement in exposure self-efficacy ( d =0.66, P <.001; d =0.93, P <.001) and beliefs ( d =0.46, P <.01; d =0.66, P <.001) from post-didactic to post-experiential. Both groups gave positive ratings for acceptability, usability, and authenticity. No adverse events or side effects were reported. Conclusions: In this feasibility randomized controlled trial, an E2E-guided VR training program produced promising improvements in therapists’ self-efficacy and negative beliefs about exposure beyond gains from didactic training alone. This work is innovative in testing immersion as a dose parameter while also applying an explicit framework (E2E) to target a key mechanism (ie, therapist beliefs) in the underuse of exposure therapy. Compared to prior VR training studies focused on skills and knowledge acquisition, our findings support the standardization of an emotionally engaging exposure practice context that shifts therapist-level mechanisms linked to actual delivery. The lack of clear advantages for HMD over desktop VR suggests that lower-immersion, more scalable implementations may provide a sufficient experiential “dose.” Larger, more diverse trials are needed to confirm effectiveness and determine the real-world impact of VR-based exposure training on access to evidence-based care.