Hongyi Yuan

Hongyi Yuan

Auto-generated profile, based on publicly available information

Hongyi Yuan

Department of Statistics and Data Science, Tsinghua University, Beijing, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 October 2023
Verified
Conference Paper
Open access
ZYZheng Yuan
Zheng Yuan
QJQiao Jin
Qiao Jin
CTChuanqi Tan
Chuanqi Tan
ZZZhengyun Zhao
Zhengyun Zhao
HYHongyi Yuan
Hongyi Yuan
SHSongfang Huang
Songfang Huang
Vision-and-language multi-modal pretraining and fine-tuning have shown great success in visual question answering (VQA). Compared to general domain VQA, the performance of biomedical VQA suffers from limited data. In this paper, we propose a retrieval-augmented pretrain-and-finetune paradigm named RAMM for biomedical VQA to overcome the data limitation issue. Specifically, we collect a new biomedical dataset named PMCPM which offers patient-based image-text pairs containing diverse patient situations from PubMed. Then, we pretrain the biomedical multi-modal model to learn visual and textual representation for image-text pairs and align these representations with image-text contrastive objective (ITC). Finally, we propose a retrieval-augmented method to better use the limited data. We propose to retrieve similar image-text pairs based on ITC from pretraining datasets and introduce a novel retrieval-attention module to fuse the representation of the image and the question with the retrieved images and texts. Experiments demonstrate that our retrieval-augmented pretrain-and-finetune paradigm obtains state-of-the-art performance on Med-VQA2019, Med-VQA2021, VQARAD, and SLAKE datasets. Further analysis shows that the proposed RAMM and PMCPM can enhance biomedical VQA performance compared with previous resources and methods. The pre-trained models and codes are published at https://github.com/GanjinZero/RAMM.

2 December 2025 in medRxiv
Verified
Preprint
Open access
PDF file
ZYZongxin Yang
Zongxin Yang
HYHongyi Yuan
Hongyi Yuan
RSRaheel Sayeed
Raheel Sayeed
ATAmelia Li Min Tan
Amelia Li Min Tan
ECEnci Cai
Enci Cai
TCTianxi Cai
Tianxi Cai
Objective: Clinical narratives in electronic health records (EHRs) hold diagnostic, therapeutic, and temporal information often missing from structured fields; manual chart review remains the de facto standard but is slow, costly, and variable. We aimed to develop and evaluate CLINES, a pipeline that extracts and normalizes clinical concepts from free-text notes into ontology-grounded, auditable, schema-ready data without task-specific fine-tuning. Methods and Analysis: We developed CLINES, a modular multi-step pipeline: semantic chunking of long notes; extraction by reasoning-capable large language models; attribute assignment (assertion, value+unit); UMLS normalization; explicit and relative date resolution; aggregation into an i2b2-style schema. Evaluation was conducted without task-specific supervised training on de-identified MIMIC-III, 4CE, and CORAL (breast, pancreas) notes. Comparators: rule/lexicon systems, transformer encoders, single-prompt LLM baselines, alongside chain-of-thought LLMs and full-size clinical NER encoders (BERT-base, GatorTron-base). Out-comes were F1 for entity, assertion, value+unit, and date processing, reported with 95% bootstrap CIs and paired-bootstrap FDR-corrected tests; inter-annotator agreement, hallucination taxonomy, cost, and component ablations are in Supplementary Materials. Results: Across all datasets and tasks, CLINES ranked first on every task-dataset combination. F1 ranges were 0.69–0.87 (entity), 0.84–0.93 (assertion), 0.77–0.90 (value+unit), and 0.73–0.79 (date, where gold annotations were available); per-dataset values are in Supplementary Materials. Gains over the strongest single-prompt LLM were +0.21–0.38; transformer encoders trailed by +0.28–0.68 F1 on entity extraction. Performance remained stable as notes lengthened, while transformer baselines lost recall. Code-field gains and most assertion/value comparisons were significant at p FDR < 0.05 on paired-bootstrap tests; mention-level inter-annotator F1 on a cross-annotated subset was 0.82 (assertion κ = 0.85); a seven-class taxonomy on 700 stratified false positives found 54% of apparent false positives to be annotation gaps rather than true hallucinations. Conclusion: CLINES translates narrative EHR text into ontology-grounded, auditable, schema-ready data, offering a practical route to scale chart-review-like extraction for cohort discovery and real-world evidence. CLINES substantially reduces, but does not eliminate, the need for manual review: the residual hallucination quantified here means human verification remains necessary for safety-critical use. CLINES is model agnostic–open and closed models can be substituted to meet cost, performance, and privacy goals. Summary box: What is already known on this topic: Free-text clinical notes hold diagnostic, temporal, and quantitative detail absent from structured EHR fields; rule-based systems and fine-tuned transformer encoders detect entities but normalize and attribute them inconsistently, and single-prompt large language models hallucinate and miss instructions on long notes. What this study adds: CLINES, a training-free multi-step large-language-model pipeline, produces UMLS-normalized, attributed, time-stamped output that substantially outperforms single-prompt and chain-of-thought LLMs (+0.37 to +0.54 code F1) and full-size clinical encoders; the gains hold across open- and closed-weight backbones and are quantified with confidence intervals, inter-annotator agreement, a hallucination taxonomy, per-note cost, and component ablations. How this study might affect research, practice, or policy: CLINES offers an auditable, model-agnostic route to scale chart-review-like extraction for cohort discovery and real-world evidence, while the quantified residual hallucination rate makes explicit that human verification remains necessary before safety-critical use.