Chunhua Weng

Chunhua Weng

Auto-generated profile, based on publicly available information

Chunhua Weng

Department of Biomedical Informatics, Columbia University, New York, USA

Research Focus & Metrics

Academic Career Length

25Years (2002-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2013 in Journal of the American Medical Informatics Association
Verified
Journal Article
Open access
PDF file
NWNicole Gray Weiskopf
Nicole Gray Weiskopf
CWChunhua Weng
Chunhua Weng
Objective To review the methods and dimensions of data quality assessment in the context of electronic health record (EHR) data reuse for research.Ma

24 August 2023 in npj Digital Medicine
Verified
Journal Article
Open access
PDF file
LTLiyan Tang
Liyan Tang
ZSZhaoyi Sun
Zhaoyi Sun
BIBetina Idnay
Betina Idnay
JNJordan G. Nestor
Jordan G. Nestor
ASAli Soroush
Ali Soroush
YPYifan Peng
Yifan Peng
Recent advances in large language models (LLMs) have demonstrated remarkable successes in zero- and few-shot performance on various downstream tasks, paving the way for applications in high-stakes domains. In this study, we systematically examine the capabilities and limitations of LLMs, specifically GPT-3.5 and ChatGPT, in performing zero-shot medical evidence summarization across six clinical domains. We conduct both automatic and human evaluations, covering several dimensions of summary quality. Our study demonstrates that automatic metrics often do not strongly correlate with the quality of summaries. Furthermore, informed by our human evaluations, we define a terminology of error types for medical evidence summarization. Our findings reveal that LLMs could be susceptible to generating factually inconsistent summaries and making overly convincing or uncertain statements, leading to potential harm due to misinformation. Moreover, we find that models struggle to identify the salient information and are more error-prone when summarizing over longer textual contexts.

11 September 2016 in eGEMs (Generating Evidence & Methods to improve patient outcomes)
Verified
Journal Article
Open access
PDF file
MKMichael G. Kahn
Michael G. Kahn
Tiffany Jean Lee CallahanTiffany Jean Lee Callahan
Tiffany Jean Lee Callahan
JBJuliana Barnard
Juliana Barnard
ABAlan E. Bauck
Alan E. Bauck
JBJeffrey Brown
Jeffrey Brown
LSLisa Schilling
Lisa Schilling
Objective: Harmonized data quality (DQ) assessment terms, methods, and reporting practices can establish a common understanding of the strengths and limitations of electronic health record (EHR) data for operational analytics, quality improvement, and research. Existing published DQ terms were harmonized to a comprehensive unified terminology with definitions and examples and organized into a conceptual framework to support a common approach to defining whether EHR data is ‘fit’ for specific uses.Materials and Methods: DQ publications, informatics and analytics experts, managers of established DQ programs, and operational manuals from several mature EHR-based research networks were reviewed to identify potential DQ terms and categories. Two face-to-face stakeholder meetings were used to vet an initial set of DQ terms and definitions that were grouped into an overall conceptual framework. Feedback received from data producers and users was used to construct a draft set of harmonized DQ terms and categories. Multiple rounds of iterative refinement resulted in a set of terms and organizing framework consisting of DQ categories, subcategories, terms, definitions, and examples. The harmonized terminology and logical framework’s inclusiveness was evaluated against ten published DQ terminologies.Results: Existing DQ terms were harmonized and organized into a framework by defining three DQ categories: (1) Conformance (2) Completeness and (3) Plausibility and two DQ assessment contexts: (1) Verification and (2) Validation. Conformance and Plausibility categories were further divided into subcategories. Each category and subcategory was defined with respect to whether the data may be verified with organizational data, or validated against an accepted gold standard, depending on proposed context and uses. The coverage of the harmonized DQ terminology was validated by successfully aligning to multiple published DQ terminologies.Discussion: Existing DQ concepts, community input, and expert review informed the development of a distinct set of terms, organized into categories and subcategories. The resulting DQ terms successfully encompassed a wide range of disparate DQ terminologies. Operational definitions were developed to provide guidance for implementing DQ assessment procedures. The resulting structure is an inclusive DQ framework for standardizing DQ assessment and reporting. While our analysis focused on the DQ issues often found in EHR data, the new terminology may be applicable to a wide range of electronic health data such as administrative, research, and patient-reported data.Conclusion: A consistent, common DQ terminology, organized into a logical framework, is an initial step in enabling data owners and users, patients, and policy makers to evaluate and communicate data quality findings in a well-defined manner with a shared vocabulary. Future work will leverage the framework and terminology to develop reusable data quality assessment and reporting methods.

7 February 2019 in Journal of the American Medical Informatics Association
Verified
Journal Article
Open access
PDF file
CYChi Yuan
Chi Yuan
PRPatrick B Ryan
Patrick B Ryan
CTCasey Ta
Casey Ta
YGYixuan Guo
Yixuan Guo
ZLZiran Li
Ziran Li
CWChunhua Weng
Chunhua Weng
Objective: Cohort definition is a bottleneck for conducting clinical research and depends on subjective decisions by domain experts. Data-driven cohort definition is appealing but requires substantial knowledge of terminologies and clinical data models. Criteria2Query is a natural language interface that facilitates human-computer collaboration for cohort definition and execution using clinical databases.Materials and Methods: Criteria2Query uses a hybrid information extraction pipeline combining machine learning and rule-based methods to systematically parse eligibility criteria text, transforms it first into a structured criteria representation and next into sharable and executable clinical data queries represented as SQL queries conforming to the OMOP Common Data Model. Users can interactively review, refine, and execute queries in the ATLAS web application. To test effectiveness, we evaluated 125 criteria across different disease domains from ClinicalTrials.gov and 52 user-entered criteria. We evaluated F1 score and accuracy against 2 domain experts and calculated the average computation time for fully automated query formulation. We conducted an anonymous survey evaluating usability.Results: Criteria2Query achieved 0.795 and 0.805 F1 score for entity recognition and relation extraction, respectively. Accuracies for negation detection, logic detection, entity normalization, and attribute normalization were 0.984, 0.864, 0.514 and 0.793, respectively. Fully automatic query formulation took 1.22 seconds/criterion. More than 80% (11+ of 13) of users would use Criteria2Query in their future cohort definition tasks.Conclusions: We contribute a novel natural language interface to clinical databases. It is open source and supports fully automated and interactive modes for autonomous data-driven cohort definition by researchers with minimal human effort. We demonstrate its promising user friendliness and usability.

1 June 2010 in Journal of Biomedical Informatics
Verified
Journal Article
Closed access
CWChunhua Weng
Chunhua Weng
STSamson W. Tu
Samson W. Tu
ISIda Sim
Ida Sim
RRRachel Richesson
Rachel Richesson

16 June 2022 in Nature Medicine
Verified
Journal Article
Closed access
PDF file
AKAtlas Khan
Atlas Khan
MTMichael C. Turchin
Michael C. Turchin
APAmit Patki
Amit Patki
VSVinodh Srinivasasainagendra
Vinodh Srinivasasainagendra
NSNing Shang
Ning Shang
KKKrzysztof Kiryluk
Krzysztof Kiryluk