Recurrence is not explicitly documented in cancer registry data that are widely used for research. Patterns of events after initial treatment such as oncology visits, re-operation, and receipt of subsequent chemotherapy or radiation may indicate recurrence. This study aimed to develop and validate algorithms for identifying breast cancer recurrence using routinely collected administrative data. The study cohort included all young (≤ 40 years) breast cancer patients (2007–2010), and all patients receiving neoadjuvant chemotherapy (2012–2014) in Alberta, Canada. Health events (including mastectomy, chemotherapy, radiation, biopsy and specialist visits) were obtained from provincial administrative data. The algorithms were developed using classification and regression tree (CART) models and validated against primary chart review. Among 598 patients, 121 (20.2%) had recurrence after a median follow-up of 4 years. The high sensitivity algorithm achieved 94.2% (95% CI: 90.1–98.4%) sensitivity, 93.7% (91.5–95.9%) specificity, 79.2% (72.5–85.8%) positive predictive value (PPV), and 98.5% (97.3–99.6%) negative predictive value (NPV). The high PPV algorithm had 75.2% (67.5–82.9%) sensitivity, 98.3% (97.2–99.5%) specificity, 91.9% (86.6–97.3%) PPV, and 94% (91.9–96.1%) NPV. Combining high PPV and high sensitivity algorithms with additional (7.5%) chart review to resolve discordant cases resulted in 94.2% (90.1–98.4%) sensitivity, 98.3% (97.2–99.5%) specificity, 93.4% (89.1–97.8%) PPV, and 98.5% (97.4–99.6%) NPV. The proposed algorithms based on routinely collected administrative data achieved favorably high validity for identifying breast cancer recurrences in a universal healthcare system in Canada.
The primary goal of this study is to evaluate the capabilities of Large Language Models (LLMs) in understanding and processing complex medical documentation. We chose to focus on the identification of pathologic complete response (pCR) in narrative pathology reports. This approach aims to contribute to the advancement of comprehensive reporting, health research, and public health surveillance, thereby enhancing patient care and breast cancer management strategies. The study utilized two analytical pipelines, developed with open-source LLMs within the healthcare system’s computing environment. First, we extracted embeddings from pathology reports using 15 different transformer-based models and then employed logistic regression on these embeddings to classify the presence or absence of pCR. Secondly, we fine-tuned the Generative Pre-trained Transformer-2 (GPT-2) model by attaching a simple feed-forward neural network (FFNN) layer to improve the detection performance of pCR from pathology reports. In a cohort of 351 female breast cancer patients who underwent neoadjuvant chemotherapy (NAC) and subsequent surgery between 2010 and 2017 in Calgary, the optimized method displayed a sensitivity of 95.3% (95%CI: 84.0–100.0%), a positive predictive value of 90.9% (95%CI: 76.5–100.0%), and an F1 score of 93.0% (95%CI: 83.7–100.0%). The results, achieved through diverse LLM integration, surpassed traditional machine learning models, underscoring the potential of LLMs in clinical pathology information extraction. The study successfully demonstrates the efficacy of LLMs in interpreting and processing digital pathology data, particularly for determining pCR in breast cancer patients post-NAC. The superior performance of LLM-based pipelines over traditional models highlights their significant potential in extracting and analyzing key clinical data from narrative reports. While promising, these findings highlight the need for future external validation to confirm the reliability and broader applicability of these methods.
Early-onset diagnosis, defined by age 60 years. The pooled odds of achieving pCR were significantly higher in early-onset patients. Future studies should evaluate the role of locoregional management of TNBC and the implementation of novel therapies such as PARP inhibitors in real-world settings, and whether they improve outcomes.
Background: In cancer survival analyses using population-based data, researchers face the challenge of ascertaining the timing of recurrence. We previously developed algorithms to identify recurrence of breast cancer. This is a follow-up study to detect the timing of recurrence. Methods: Health events that signified recurrence and timing were obtained from routinely collected administrative data. The timing of recurrence was estimated by finding the timing of key indicator events using three different algorithms, respectively. For validation, we compared algorithm-estimated timing of recurrence with that obtained from chart-reviewed data. We further compared the results of cox regressions models (modeling recurrence-free survival) based on the algorithms versus chart review. Results: In total, 598 breast cancer patients were included. 121 (20.2%) had recurrence after a median follow-up of 4 years. Based on the high accuracy algorithm for identifying the presence of recurrence (with 94.2% sensitivity and 79.2% positive predictive value), the majority (64.5%) of the algorithm-estimated recurrence dates fell within 3 months of the corresponding chart review determined recurrence dates. The algorithm estimated and chart-reviewed data generated Kaplan–Meier (K-M) curves and Cox regression results for recurrence-free survival (hazard ratios and P-values) were very similar. Conclusion: The proposed algorithms for identifying the timing of breast cancer recurrence achieved similar results to the chart review data and were potentially useful in survival analysis.
Background: Inpatient falls are a substantial concern for health care providers and are associated with negative outcomes for patients. Automated detection of falls using machine learning (ML) algorithms may aid in improving patient safety and reducing the occurrence of falls. Objective: This study aims to develop and evaluate an ML algorithm for inpatient fall detection using multidisciplinary progress record notes and a pretrained Bidirectional Encoder Representation from Transformers (BERT) language model. Methods: A cohort of 4323 adult patients admitted to 3 acute care hospitals in Calgary, Alberta, Canada from 2016 to 2021 were randomly sampled. Trained reviewers determined falls from patient charts, which were linked to electronic medical records and administrative data. The BERT-based language model was pretrained on clinical notes, and a fall detection algorithm was developed based on a neural network binary classification architecture. Results: To address various use scenarios, we developed 3 different Alberta hospital notes-specific BERT models: a high sensitivity model (sensitivity 97.7, IQR 87.7-99.9), a high positive predictive value model (positive predictive value 85.7, IQR 57.2-98.2), and the high F1-score model (F1=64.4). Our proposed method outperformed 3 classical ML algorithms and an International Classification of Diseases code–based algorithm for fall detection, showing its potential for improved performance in diverse clinical settings. Conclusions: The developed algorithm provides an automated and accurate method for inpatient fall detection using multidisciplinary progress record notes and a pretrained BERT language model. This method could be implemented in clinical practice to improve patient safety and reduce the occurrence of falls in hospitals.