Hanjae Kim

Hanjae Kim

Auto-generated profile, based on publicly available information

Hanjae Kim

Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Republic of Korea

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024 in Journal of Korean Medical Science
Verified
Journal Article
Open access
PDF file
HKHanjae Kim
Hanjae Kim
HJHee Min Jin
Hee Min Jin
YJYoon Bin Jung
Yoon Bin Jung
SYSeng Chan You
Seng Chan You
Kim H, et al. J Korean Med Sci. 2024 Apr;39(16):e148. https://doi.org/10.3346/jkms.2024.39.e148

24 November 2025 in JMIR Medical Informatics
Verified
Journal Article
Open access
HKHanjae Kim
Hanjae Kim
SLSo-Yeon Lee
So-Yeon Lee
SYSeng Chan You
Seng Chan You
SHSookyung Huh
Sookyung Huh
Jai-Eun KimJai-Eun Kim
Jai-Eun Kim
KLKang Young Lee
Kang Young Lee
Background: Large language models (LLMs) have shown promise in reducing clinical documentation burden, yet their real-world implementation remains rare. Especially in South Korea, hospitals face several unique challenges, such as strict data sovereignty requirements and operating in environments where English is not the primary language for documentation. Therefore, we initiated the Your-Knowledgeable Navigator of Treatment (Y-KNOT) project, aimed at developing an on-premises bilingual LLM-based artificial intelligence (AI) agent system integrated with electronic health records (EHRs) for automated clinical drafting. Objective: We present the Y-KNOT project and provide insights into implementing AI-assisted clinical drafting tools within constraints of health care system. Methods: This project involved multiple stakeholders and encompassed three simultaneous processes: LLM development, clinical co-development, and EHR integration. We developed a foundation LLM by pretraining Llama3-8B with Korean and English medical corpora. During the clinical co-development phase, the LLM was instruction-tuned for specific documentation tasks through iterative cycles that aligned physicians’ clinical requirements, hospital data availability, documentation standards, and technical feasibility. The EHR integration phase focused on seamless AI agent incorporation into clinical workflows, involving document standardization, trigger points definition, and user interaction optimization. Implementation (Results): The resulting system processes emergency department discharge summaries and preanesthetic assessments while maintaining existing clinical workflows. The drafting process is automatically triggered by specific events, such as scheduled batch jobs, with medical records automatically fed into the LLM as input. The agent is built on premises, locating all the architecture inside the hospital. Conclusions: The Y-KNOT project demonstrates the first seamless integration of an AI agent into an EHR system for clinical drafting. In collaboration with various clinical and administrative teams, we could promptly implement an LLM while addressing key challenges of data security, bilingual requirements, and workflow integration. Our experience highlights a practical and scalable approach to utilizing LLM-based AI agents for other health care institutions, paving the way for broader adoption of LLM-based solutions.

4 April 2025 in medRxiv
Verified
Preprint
Open access
PDF file
HKHanjae Kim
Hanjae Kim
SLSo-Yeon Lee
So-Yeon Lee
SYSeng Chan You
Seng Chan You
SHSookyung Huh
Sookyung Huh
Jai-Eun KimJai-Eun Kim
Jai-Eun Kim
KLKang Young Lee
Kang Young Lee
Large Language Models (LLMs) have shown promise in reducing clinical documentation burden, yet their real-world implementation faces significant challenges, particularly in non-English speaking countries with strict data sovereignty requirements. Here we present Your-Knowledgeable Navigator of Treatment (Y-KNOT), the first successful implementation of an on-premise bilingual LLM-based artificial intelligence system integrated with electronic health records (EHR) for automated clinical documentation. In collaboration with multiple stakeholders, we developed and deployed Y-KNOT at a tertiary hospital in South Korea. The system processes emergency department discharge summaries and pre-anesthetic assessments with high evaluation scores across multiple clinical metrics while maintaining FHIR compliance for scalability. Our study demonstrates a practical framework for implementing LLM-based clinical documentation systems in resource-constrained healthcare settings while addressing key challenges of data security, bilingual requirements, and workflow integration.

14 March 2026 in medRxiv
Verified
Preprint
Open access
PDF file
HKHanjae Kim
Hanjae Kim
MKMinseong Kim
Minseong Kim
SKSeonji Kim
Seonji Kim
SYSeng Chan You
Seng Chan You
Introduction: Implementing target trial emulation (TTE) study methods as end-to-end executable analytic code is technically demanding, and producing standardized, reproducible scripts consistently across research teams remains a persistent challenge. We aimed to develop a framework that translates free-text study descriptions into standardized analytic specifications and executable Strategus R scripts for the Observational Health Data Sciences and Informatics (OHDSI) ecosystem. Methods: We developed THESEUS (Text-guided Health-study Estimation and Specification Engine Using Strategus), which operates through two sequential steps. Large language models (LLMs) first map descriptions of the study into a constrained JavaScript Object Notation (JSON) schema (standardization step), after which the structured specifications are converted into R scripts with a self-auditing loop for error correction (code generation step). We evaluated eight proprietary LLMs using texts extracted from the methods section of 15 OHDSI-based TTE studies, and externally validated the framework on texts from 5 non-OHDSI studies, across three input settings: primary analysis text only, full analyses text, and full methods sections. Standardization was evaluated at the study-level (whether all parameters in a study were correctly extracted) and at the field-level (sensitivity and false positive rate per individual parameter) with field-level evaluation applied to the full analyses text and full methods sections input settings. Code generation was assessed by executability of the produced R scripts before and after self-auditing. Results: In the standardization step, study-level accuracy across models ranged from 0.91 to 0.98 for primary analysis, 0.67 to 0.87 for full analyses, and 0.67 to 0.85 for full methods sections in OHDSI studies, whereas the corresponding ranges were 0.73 to 0.93, 0.60 to 0.87, and 0.27 to 0.47 in non-OHDSI studies. At the field-level, sensitivity across models under the full analyses text input setting ranged from 0.73 to 0.90 with 0.27 to 0.67 false positives per study in OHDSI studies, and from 0.71 to 0.90 with 0.20 to 1.00 false positives per study in non-OHDSI studies, depending on input setting. For code generation, first-run executability ranged from 0.80 to 1.00 for OHDSI studies and improved to 0.93 to 1.00 after self-auditing. In non-OHDSI studies, first-run executability ranged from 0.60 to 1.00, improving to 1.00 after self-auditing. Discussion: THESEUS demonstrates that pairing a standardized data model with a structured analysis framework enables reliable LLM-powered automation of the coding step in observational research. THESEUS supports the reliable translation of natural-language study descriptions into executable, shareable code in standardized observational research settings. This approach has the potential to lower the technical barriers to participation in observational research for a broader range of investigators.