Jie Pan

Jie Pan

Auto-generated profile, based on publicly available information

Jie Pan

Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 May 2025 in BMJ Health & Care Informatics
Verified
Journal Article
Open access
PDF file
JPJie Pan
Jie Pan
L(Lee Seungwon (Frank)
Lee Seungwon (Frank)
CCCheligeer Cheligeer
Cheligeer Cheligeer
BLBing Li
Bing Li
GWGuosong Wu
Guosong Wu
HQHude Quan
Hude Quan
Objectives: Administrative data are commonly used to inform chronic disease prevalence and support health informatic research. This study assessed the validity of coding comorbidities in the International Classification of Diseases, 10th Revision (ICD-10) administrative data. Methods: We analysed three chart review cohorts (4008 patients in 2003, 3045 in 2015 and 9024 in 2022) in Alberta, Canada. Nurse reviewers assessed the presence of 17 clinical conditions using a consistent protocol. The reviews were linked with administrative data using unique patient identifiers. We compared the accuracy in coding comorbidity by ICD-10, using chart review data as the reference standard. Results: Our findings showed that the mean difference in prevalence between chart reviews and ICD-10 for these 17 conditions was 2.1% in 2003, 7.6% in 2015 and 6.3% in 2022. Some conditions were relatively stable, such as diabetes (1.9%, 2.1% and 1.1%) and metastatic cancer (0.3%, 1.1% and 0.4%). For these 17 conditions, the sensitivity ranged from 39.6–85.1% in 2003, 1.3%–85.2% in 2015 and 3.0–89.7% in 2022. The C-statistics for predicting in-hospital mortality using comorbidities by ICD-10 were 0.84 in 2003, 0.81 in 2015 and 0.78 in 2022. Discussion: The undercoding could be primarily due to the increase in hospital patient volumes and the limited time allocated to coding specialists. There is the potential to develop artificial intelligence methods based on electronic health records to support coding practices and improve data quality. Conclusion: Comorbidities were increasingly undercoded over 20 years. The validity of ICD-10 decreased but remained relatively stable for certain conditions mandated for coding. The undercoding exerted minimal impact on in-hospital mortality prediction.

30 January 2024 in JMIR Medical Informatics
Verified
Journal Article
Open access
CCCheligeer Cheligeer
Cheligeer Cheligeer
GWGuosong Wu
Guosong Wu
L(Lee Seungwon (Frank)
Lee Seungwon (Frank)
JPJie Pan
Jie Pan
DSDanielle A Southern
Danielle A Southern
YXYuan Xu
Yuan Xu
Background: Inpatient falls are a substantial concern for health care providers and are associated with negative outcomes for patients. Automated detection of falls using machine learning (ML) algorithms may aid in improving patient safety and reducing the occurrence of falls. Objective: This study aims to develop and evaluate an ML algorithm for inpatient fall detection using multidisciplinary progress record notes and a pretrained Bidirectional Encoder Representation from Transformers (BERT) language model. Methods: A cohort of 4323 adult patients admitted to 3 acute care hospitals in Calgary, Alberta, Canada from 2016 to 2021 were randomly sampled. Trained reviewers determined falls from patient charts, which were linked to electronic medical records and administrative data. The BERT-based language model was pretrained on clinical notes, and a fall detection algorithm was developed based on a neural network binary classification architecture. Results: To address various use scenarios, we developed 3 different Alberta hospital notes-specific BERT models: a high sensitivity model (sensitivity 97.7, IQR 87.7-99.9), a high positive predictive value model (positive predictive value 85.7, IQR 57.2-98.2), and the high F1-score model (F1=64.4). Our proposed method outperformed 3 classical ML algorithms and an International Classification of Diseases code–based algorithm for fall detection, showing its potential for improved performance in diverse clinical settings. Conclusions: The developed algorithm provides an automated and accurate method for inpatient fall detection using multidisciplinary progress record notes and a pretrained BERT language model. This method could be implemented in clinical practice to improve patient safety and reduce the occurrence of falls in hospitals.

2 September 2023 in Brain Informatics
Verified
Journal Article
Open access
PDF file
JPJie Pan
Jie Pan
ZZZilong Zhang
Zilong Zhang
SPSteven Ray Peters
Steven Ray Peters
SVShabnam Vatanpour
Shabnam Vatanpour
RWRobin L. Walker
Robin L. Walker
HQHude Quan
Hude Quan
Background: Abstracting cerebrovascular disease (CeVD) from inpatient electronic medical records (EMRs) through natural language processing (NLP) is pivotal for automated disease surveillance and improving patient outcomes. Existing methods rely on coders’ abstraction, which has time delays and under-coding issues. This study sought to develop an NLP-based method to detect CeVD using EMR clinical notes. Methods: CeVD status was confirmed through a chart review on randomly selected hospitalized patients who were 18 years or older and discharged from 3 hospitals in Calgary, Alberta, Canada, between January 1 and June 30, 2015. These patients’ chart data were linked to administrative discharge abstract database (DAD) and Sunrise™ Clinical Manager (SCM) EMR database records by Personal Health Number (a unique lifetime identifier) and admission date. We trained multiple natural language processing (NLP) predictive models by combining two clinical concept extraction methods and two supervised machine learning (ML) methods: random forest and XGBoost. Using chart review as the reference standard, we compared the model performances with those of the commonly applied International Classification of Diseases (ICD-10-CA) codes, on the metrics of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Result: Of the study sample (n = 3036), the prevalence of CeVD was 11.8% (n = 360); the median patient age was 63; and females accounted for 50.3% (n = 1528) based on chart data. Among 49 extracted clinical documents from the EMR, four document types were identified as the most influential text sources for identifying CeVD disease (“nursing transfer report,” “discharge summary,” “nursing notes,” and “inpatient consultation.”). The best performing NLP model was XGBoost, combining the Unified Medical Language System concepts extracted by cTAKES (e.g., top-ranked concepts, “Cerebrovascular accident” and “Transient ischemic attack”), and the term frequency-inverse document frequency vectorizer. Compared with ICD codes, the model achieved higher validity overall, such as sensitivity (25.0% vs 70.0%), specificity (99.3% vs 99.1%), PPV (82.6 vs. 87.8%), and NPV (90.8% vs 97.1%). Conclusion: The NLP algorithm developed in this study performed better than the ICD code algorithm in detecting CeVD. The NLP models could result in an automated EMR tool for identifying CeVD cases and be applied for future studies such as surveillance, and longitudinal studies.

8 April 2025 in Computers in Biology and Medicine
Verified
Journal Article
Open access
JPJie Pan
Jie Pan
L(Lee Seungwon (Frank)
Lee Seungwon (Frank)
CCCheligeer Cheligeer
Cheligeer Cheligeer
EMElliot Martin
Elliot Martin
KRKiarash Riazi
Kiarash Riazi
NLNa Li
Na Li
Electronic health records (EHR) are widely available to complement administrative data-based disease surveillance and healthcare performance evaluation. Defining conditions from EHR is labour-intensive and requires extensive manual labelling of disease outcomes. This study developed an efficient strategy based on advanced large language models to identify multiple conditions from EHR clinical notes. We linked a cardiac registry cohort in 2015 with an EHR system in Alberta, Canada. We developed a pipeline that leveraged a generative large language model (LLM) to analyze, understand, and interpret EHR notes by prompts based on specific diagnosis, treatment management, and clinical guidelines. The pipeline was applied to detect acute myocardial infarction (AMI), diabetes, and hypertension. The performance was compared against clinician-validated diagnoses as the reference standard and widely adopted International Classification of Diseases (ICD) codes-based methods. The study cohort accounted for 3088 patients and 551,095 clinical notes. The prevalence was 55.4 %, 27.7 %, 65.9 % and for AMI, diabetes, and hypertension, respectively. The performance of the LLM-based pipeline for detecting conditions varied: AMI had 88 % sensitivity, 63 % specificity, and 77 % positive predictive value (PPV); diabetes had 91 % sensitivity, 86 % specificity, and 71 % PPV; and hypertension had 94 % sensitivity, 32 % specificity, and 72 % PPV. Compared with ICD codes, the LLM-based method demonstrated improved sensitivity and negative predictive value across all conditions. The monthly percentage trends from the detected cases by LLM and reference standard showed consistent patterns. The proposed LLM-based pipeline demonstrated reasonable accuracy and high efficiency in disease detection for multiple conditions. Human expert knowledge can be integrated into the pipeline to guide EHR note analysis without manually curated labels. The method could enable comprehensive real-time disease surveillance using EHRs.

25 February 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
JYJinli Yao
Jinli Yao
JPJie Pan
Jie Pan
YZYong Zeng
Yong Zeng
The development of a nonparametric and versatile clustering algorithm has been a longstanding challenge in unsupervised learning due to the exploratory nature of the clustering problem. This study presents a novel algorithm, named Gauging-$\delta$, which can handle diverse cluster shapes and operate in a nonparametric manner. The algorithm employs a hierarchical merging process that starts from individual data points until no further clusters can be merged. The central component of Gauging-$\delta$ is the adaptive mergeability function, which progressively determines if two clusters are mergeable considering the perceptual statistics of the clusters and their environment. Empirical evaluations on 105 synthetic datasets demonstrate the superiority of the proposed algorithm, particularly in accurately handling well-separated clusters. Experiments on real-world datasets highlight the impact of selecting appropriate data features and distance metrics on clustering results.

19 March 2025 in Journal of the American Medical Informatics Association
Verified
Journal Article
Open access
CCCheligeer Cheligeer
Cheligeer Cheligeer
DSDanielle A Southern
Danielle A Southern
JYJun Yan
Jun Yan
GWGuosong Wu
Guosong Wu
JPJie Pan
Jie Pan
HQHude Quan
Hude Quan
Objectives: Adverse event detection from Electronic Medical Records (EMRs) is challenging due to the low incidence of the event, variability in clinical documentation, and the complexity of data formats. Pulmonary embolism as an adverse event (PEAE) is particularly difficult to identify using existing approaches. This study aims to develop and evaluate a Large Language Model (LLM)-based framework for detecting PEAE from unstructured narrative data in EMRs. Materials and Methods: We conducted a chart review of adult patients (aged 18-100) admitted to tertiary-care hospitals in Calgary, Alberta, Canada, between 2017-2022. We developed an LLM-based detection framework consisting of three modules: evidence extraction (implementing both keyword-based and semantic similarity-based filtering methods), discharge information extraction (focusing on six key clinical sections), and PEAE detection. Four open-source LLMs (Llama3, Mistral-7B, Gemma, and Phi-3) were evaluated using positive predictive value, sensitivity, specificity, and F1-score. Model performance for population-level surveillance was assessed at yearly, quarterly, and monthly granularities. Results: The chart review included 10 066 patients, with 40 cases of PEAE identified (0.4% prevalence). All four LLMs demonstrated high sensitivity (87.5-100%) and specificity (94.9-98.9%) across different experimental conditions. Gemma achieved the highest F1-score (28.11%) using keyword-based retrieval with discharge summary inclusion, along with 98.4% specificity, 87.5% sensitivity, and 99.95% negative predictive value. Keyword-based filtering reduced the median chunks per patient from 789 to 310, while semantic filtering further reduced this to 9 chunks. Including discharge summaries improved performance metrics across most models. For population-level surveillance, all models showed strong correlation with actual PEAE trends at yearly granularity (r=0.92-0.99), with Llama3 achieving the highest correlation (0.988). Discussion: The results of our method for PEAE detection using EMR notes demonstrate high sensitivity and specificity across all four tested LLMs, indicating strong performance in distinguishing PEAE from non-PEAE cases. However, the low incidence rate of PEAE contributed to a lower PPV. The keyword-based chunking approach consistently outperformed semantic similarity-based methods, achieving higher F1 scores and PPV, underscoring the importance of domain knowledge in text segmentation. Including discharge summaries further enhanced performance metrics. Our population-based analysis revealed better performance for yearly trends compared to monthly granularity, suggesting the framework's utility for long-term surveillance despite dataset imbalance. Error analysis identified contextual misinterpretation, terminology confusion, and preprocessing limitations as key challenges for future improvement. Conclusions: Our proposed method demonstrates that LLMs can effectively detect PEAE from narrative EMRs with high sensitivity and specificity. While these models serve as effective screening tools to exclude non-PEAE cases, their lower PPV indicates they cannot be relied upon solely for definitive PEAE identification. Further chart review remains necessary for confirmation. Future work should focus on improving contextual understanding, medical terminology interpretation, and exploring advanced prompting techniques to enhance precision in adverse event detection from EMRs.