Yuhao Chen

Yuhao Chen

Auto-generated profile, based on publicly available information

Yuhao Chen

Queen’s University - Kingston Campus: Queen’s University, Queen’s University, Kingston, Canada

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 December 2025 in JMIR AI
Verified
Journal Article
Open access
YCYuhao Chen
Yuhao Chen
BWBo Wen
Bo Wen
FZFarhana Zulkernine
Farhana Zulkernine
Background: Although large language models (LLMs) show great promise in processing medical text, they are prone to generating incorrect information, commonly referred to as hallucinations. These inaccuracies present a significant risk for clinical applications where precision is critical. Additionally, relying on human experts to review LLM-generated content to ensure accuracy is costly and time-consuming, which sets a barrier against large-scale deployment of LLMs in health care settings. Objective: The primary objective of this study was to develop an automatic artificial intelligence (AI) system capable of extracting structured information from unstructured medical data and using advanced reasoning techniques to support reliable clinical decision making. A key aspect of this objective is ensuring that the system incorporates self-verification mechanisms, enabling it to assess the accuracy and reliability of its own outputs. By integrating such mechanisms, we aim to enhance the system’s robustness, reduce reliance on human intervention, and improve the overall trustworthiness of AI-driven medical summarization and evaluation. Methods: The proposed framework comprises 2 layers: a summarization layer and an evaluation layer. The summarization layer uses Llama2-70B (Meta AI) and Mistral-7B (Mistral AI) models to generate concise summaries from unstructured medical data, focusing on tasks such as consumer health question summarization, biomedical answer summarization, and dialog summarization. The evaluation layer uses GPT-4-turbo (OpenAI) as a judge, leveraging pairwise comparison strategies and different prompt strategies to evaluate summaries across 4 dimensions: coherence, consistency, fluency, and relevance. To validate the framework, we compare the judgments generated by the LLM assistants in the evaluation layer with those provided by medical experts, offering valuable insights into the alignment and reliability of AI-driven evaluations within the medical domain. We also explore a way to handle disagreement among human experts and discuss our methodology in addressing diversity in human perspectives. Results: The study found variability in expert consensus, with average agreement rates of 19.2% among all experts and 54% among groups of 3 experts. GPT-4 (OpenAI) demonstrated alignment with expert judgments, achieving an average agreement rate of 83.06% with at least 1 expert and comparable performance in cross-validation tests. The enhanced guidance in prompt design (prompt-enhanced guidance) improved GPT-4’s alignment with expert evaluations compared with a baseline prompt, highlighting the importance of effective prompt engineering in auto-evaluation of summarization tasks. We also evaluated open-source LLMs, including Llama-3.3 (Meta AI) and Mixtral-Large (Mistral AI), and a domain-specific LLM, OpenBioLLM (Aaditya Ura), for comparison as LLM judges. Conclusions: This study highlights the potential of LLMs as reliable tools for unstructured medical data summarization and evaluation to reduce the dependency on human experts and also states the limitations. The proposed framework, multiagent summarization and auto-evaluation, demonstrates scalability and adaptability for clinical applications while addressing key challenges like hallucination and position bias.

8 April 2025 in Journal of Computational and Cognitive Engineering
Verified
Journal Article
Open access
PDF file
JLJason Lam
Jason Lam
YCYuhao Chen
Yuhao Chen
FZFarhana Zulkernine
Farhana Zulkernine
SDSamuel Dahan
Samuel Dahan
Applications of deep learning (DL) to generate text embeddings and natural language processing (NLP) have shown wide success in semantic interpretations of domain-specific text data when applied to downstream tasks such as predicting the next word, information extraction for classification, analyzing social media feeds, classifying text, and creating compressed representations. While DL and NLP have been widely applied across numerous domains, researchers have recently begun to apply these techniques to the field of law due to the challenges in processing legal case descriptions. Attention-based models have shown promising results in predicting criminal charges using unstructured text as an input, but little work has been done on data representing the Canadian legal system, especially employment law. The legal field poses many challenges, such as the amount of legal data publicly available in Canada, the verbosity of judgments, the legal jargon used in judgments, and the subjectivity of outcomes that pose many challenges in processing legal text data. Many of the state-of-the-art systems require expensive hand-annotated labels that are often unobtainable. In this study, we investigate the prediction of reasonable notice for termination of employment in the field of law. To address these challenges, we propose domain-adapted BERT variations specifically trained for legal texts. We assess the performance of various attention-based and pre-trained models using human-typed summaries of legal judgment and present a detailed analysis of the data and the results to provide insights for further exploration in this area. Our approaches also provide interesting insights about this specific type of legal case focusing on employment law, given the subjective nature of judges and the variability in outcomes. Received: 15 August 2024 | Revised: 28 February 2025 | Accepted: 10 March 2025 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available at https://static.case.law/. Author Contribution Statement Jason Lam: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Yuhao Chen: Writing – review & editing, Visualization. Farhana Zulkernine: Conceptualization, Resources, Data curation, Writing – review & editing, Visualization, Supervision, Project administration, Funding acquisition. Samuel Dahan: Validation, Data curation, Writing – review & editing, Supervision, Project administration, Funding acquisition.

15 April 2025 in JMIR Preprints
Verified
Preprint
Open access
YCYuhao Chen
Yuhao Chen
BWBo Wen
Bo Wen
FZFarhana Zulkernine
Farhana Zulkernine
BACKGROUND Although Large Language Models (LLMs) show great promises in processing medical text, they are prone to generating incorrect information, commonly referred to as hallucinations. These inaccuracies present a significant risk for clinical applications where precision is critical. Additionally, relying on human experts to review LLM-generated content to ensure accuracy is costly and time-consuming, which sets a barrier against large-scale deployment of LLMs in healthcare settings. OBJECTIVE The primary objective of this study is to develop an automatic Artificial Intelligence (AI) system capable of extracting structured information from unstructured medical data and employing advanced reasoning techniques to support reliable clinical decision making. A key aspect of this objective is ensuring that the system incorporates self-verification mechanisms, enabling it to assess the accuracy and reliability of its own outputs. By integrating such mechanisms, we aim to enhance the system’s robustness, reduce reliance on human intervention, and improve the overall trustworthiness of AI-driven medical summarization and evaluation. METHODS The proposed framework comprises two layers: a summarization layer and an evaluation layer. The summarization layer employs Llama2-70B and Mistral-7B models to generate concise summaries from unstructured medical data, focusing on tasks such as consumer health question summarization, biomedical answer summarization, and dialog summarization. The evaluation layer uses GPT-4-turbo as a judge, leveraging pairwise comparison strategies and different prompt strategies to evaluate summaries across four dimensions: coherence, consistency, fluency, and relevance. To validate the framework, we compare the judgments generated by the LLMs in the evaluation layer with those provided by medical experts, offering valuable insights into the alignment and reliability of AI-driven evaluations within the medical domain. We also explore a way to handle disagreement among human experts and discuss our methodology in addressing diversity in human perspectives. RESULTS The study found variability in expert consensus, with average Agreement Rates (ARs) of 19.2% among all experts and 51.6% among groups of three experts. GPT-4 demonstrated alignment with expert judgments, achieving an average AR of 78.44% with at least one expert and comparable performance in cross-validation tests. The enhanced guidance in prompt design improved GPT-4’s alignment with expert evaluations, highlighting the importance of effective prompt engineering in auto-evaluation of summarization tasks. CONCLUSIONS This study highlights the potential of LLMs as reliable tools for medical summarization and evaluation, reducing the dependency on human experts. The proposed framework demonstrates scalability and adaptability for clinical applications while addressing key challenges like hallucination and position bias. INTERNATIONAL REGISTERED REPORT RR2-https://doi.org/10.1109/ICDH62654.2024.00030

5 August 2026 in Intelligent Data Analysis: An International Journal
Verified
Journal Article
Open access
YCYuhao Chen
Yuhao Chen
JCJiahao Cai
Jiahao Cai
NSNafiz Sadman
Nafiz Sadman
FZFarhana Zulkernine
Farhana Zulkernine
JQJohn Queenan
John Queenan
DBDavid Barber
David Barber
Osteoarthritis (OA) is a progressive chronic joint disease resulting in a breakdown of articular cartilage and bone when damaged joint tissues are not able to normally repair themselves. The aim of this pilot research study is to understand the pain severity for OA from patients’ primary care Electronic Medical Records (EMR), both from the structured medical data and the unstructured chart note data using information extraction, natural language processing and machine learning techniques. We propose SPaDe, a Synonym-based Pain level Detection tool to categorize patients into having mild or moderate-to-severe pain to understand diagnosis and treatment methods based on only the pain related expressions in the unstructured chart note. Expressions are subjective, objective, and influenced by cultural background and demography which poses a difficult challenge. Therefore, we improve the model by incorporating the medication information from the structured EMR data and pain scale related information from the chart note to propose an integrated pain level detection tool for OA called PLeDO. With the help of human labeled gold standard data, we demonstrate that both SPaDe and PLeDO can detect mild and moderate-to-severe pain from the EMR data to analyze and potentially improve the quality of care in primary care setting.