Aaron Benjamin Cohen

Dr. Aaron Benjamin Cohen

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 January 2025 in ESMO Open
Verified
Journal Article
Open access
HRHope Rugo
Hope Rugo
RLRachel M Layman
Rachel M Layman
FLFilipa Lynce
Filipa Lynce
Xianchen LiuXianchen Liu
Xianchen Liu
BLB. Li
B. Li
ABAdam Brufsky
Adam Brufsky
Randomized controlled trials have shown inconsistent overall survival (OS) benefit among the three cyclin-dependent kinase 4/6 inhibitors (CDK4/6i) as first-line (1L) treatment of patients with hormone receptor-positive (HR+)/human epidermal growth factor receptor 2-negative (HER2-) metastatic breast cancer (mBC). Several real-world studies compared CDK4/6i effectiveness, with inconsistent findings. This study compared overall survival (OS) of patients with HR+/HER2- mBC receiving 1L palbociclib, ribociclib, or abemaciclib, in combination with an aromatase inhibitor (AI), in US clinical practice. This retrospective study used real-world data from the Flatiron Health electronic health record-derived deidentified longitudinal database. Patients with HR+/HER2- mBC aged ≥18 years at mBC diagnosis started 1L CDK4/6i therapy (index treatment) between February 2015 and November 2023, with a potential ≥6-month follow-up. OS was defined as months from start of index treatment to death. Stabilized inverse probability of treatment weighting (sIPTW; primary analysis) was used to balance baseline patient characteristics. Multivariable Cox proportional hazards model was carried out as a sensitivity analysis. Of 9146 eligible patients, 6831, 1279, and 1036 received palbociclib plus AI, ribociclib plus AI, or abemaciclib plus AI, respectively. After sIPTW, baseline characteristics were balanced between treatment groups. After sIPTW, no significant OS differences were found between treatment groups [ribociclib versus palbociclib: adjusted hazard ratio (aHR) 0.98, 95% confidence interval (CI) 0.87-1.10, P = 0.7531; abemaciclib versus palbociclib: aHR 0.95, 95% CI 0.84-1.08, P = 0.4292; abemaciclib versus ribociclib: aHR 0.97, 95% CI 0.82-1.14, P = 0.6956]. Sensitivity analysis including a subanalysis of patients who started index treatment in 2017 or later also showed no significant OS differences between treatment groups. This large real-world study suggested that there were no significant OS differences between 1L ribociclib, abemaciclib, and palbociclib in combination with an AI for patients with HR+/HER2- mBC. These findings together with other factors such as safety and quality of life are helpful in the selection of CDK4/6i combination therapy for patients with HR+/HER2- mBC.

22 June 2022 in Cancers
Verified
Journal Article
Open access
PDF file
MEMelissa Estevez
Melissa Estevez
CBCorey Benedum
Corey Benedum
CJChengsheng Jiang
Chengsheng Jiang
ACAaron Benjamin Cohen
Aaron Benjamin Cohen
SPSharang Phadke
Sharang Phadke
Selen BozkurtSelen Bozkurt
Selen Bozkurt
A vast amount of real-world data, such as pathology reports and clinical notes, are captured as unstructured text in electronic health records (EHRs). However, this information is both difficult and costly to extract through human abstraction, especially when scaling to large datasets is needed. Fortunately, Natural Language Processing (NLP) and Machine Learning (ML) techniques provide promising solutions for a variety of information extraction tasks such as identifying a group of patients who have a specific diagnosis, share common characteristics, or show progression of a disease. However, using these ML-extracted data for research still introduces unique challenges in assessing validity and generalizability to different cohorts of interest. In order to enable effective and accurate use of ML-extracted real-world data (RWD) to support research and real-world evidence generation, we propose a research-centric evaluation framework for model developers, ML-extracted data users and other RWD stakeholders. This framework covers the fundamentals of evaluating RWD produced using ML methods to maximize the use of EHR data for research purposes.

18 May 2023 in Applied Sciences
Verified
Journal Article
Open access
PDF file
ACAaron Benjamin Cohen
Aaron Benjamin Cohen
ARAndrej Rosic
Andrej Rosic
KHKatherine Harrison
Katherine Harrison
MRMadeline Richey
Madeline Richey
SNSheila Nemeth
Sheila Nemeth
CJChengsheng Jiang
Chengsheng Jiang
Our goal was to develop and characterize a Natural Language Processing (NLP) algorithm to extract Eastern Cooperative Oncology Group Performance Status (ECOG PS) from unstructured electronic health record (EHR) sources to enhance observational datasets. By scanning unstructured EHR-derived documents from a real-world database, the NLP algorithm assigned ECOG PS scores to patients diagnosed with one of 21 cancer types who lacked structured ECOG PS numerical scores, anchored to the initiation of treatment lines. Manually abstracted ECOG PS scores were used as a source of truth to both develop the algorithm and evaluate accuracy, sensitivity, and positive predictive value (PPV). Algorithm performance was further characterized by investigating the prognostic value of composite ECOG PS scores in patients with advanced non-small cell lung cancer receiving first line treatment. Of N = 480,825 patient-lines, structured ECOG PS scores were available for 290,343 (60.4%). After applying NLP-extraction, the availability increased to 73.2%. The algorithm’s overall accuracy, sensitivity, and PPV were 93% (95% CI: 92–94%), 88% (95% CI: 87–89%), and 88% (95% CI: 87–89%), respectively across all cancer types. In a cohort of N = 51,948 aNSCLC patients receiving 1L therapy, the algorithm improved ECOG PS completeness from 61.5% to 75.6%. Stratification by ECOG PS showed worse real-world overall survival (rwOS) for patients with worse ECOG PS scores. We developed an NLP algorithm to extract ECOG PS scores from unstructured EHR documents with high accuracy, improving data completeness for EHR-derived oncology cohorts.

1 January 2026 in JCO Clinical Cancer Informatics
Verified
Journal Article
Open access
MEMelissa Estevez
Melissa Estevez
NSNisha Singh
Nisha Singh
LDLauren Dyson
Lauren Dyson
Blythe AdamsonBlythe Adamson
Blythe Adamson
KKKonstantin Krismer
Konstantin Krismer
ACAaron Benjamin Cohen
Aaron Benjamin Cohen
Large language models (LLMs) are increasingly used to extract clinical data from electronic health records, offering significant improvements in scalability and efficiency for real-world data (RWD) curation in oncology. However, the adoption of LLMs introduces new challenges in ensuring the reliability, accuracy, and fairness of extracted data, which are essential for research, regulatory, and clinical applications. Existing quality assurance frameworks for RWD and artificial intelligence (AI) do not fully address the unique error modes and complexities associated with LLM-extracted data. In this paper, we propose a comprehensive framework for evaluating the quality of clinical data extracted by LLMs. The framework integrates variable-level performance benchmarking against expert human abstraction, verification checks for internal consistency and plausibility, and replication analyses comparing LLM-extracted data to human-abstracted data sets or external standards. This multidimensional approach enables the identification of variables most in need of improvement, systematic detection of latent errors, and confirmation of data set fitness-for-purpose in real-world research. Additionally, the framework supports bias assessment by stratifying across demographic subgroups. By providing a rigorous and transparent method for assessing LLM-extracted RWD, this framework advances industry standards and supports the trustworthy use of AI-powered evidence generation in oncology research and practice.

1 April 2025 in AI in Precision Oncology
Verified
Journal Article
Open access
ACAaron Benjamin Cohen
Aaron Benjamin Cohen
Blythe AdamsonBlythe Adamson
Blythe Adamson
JLJonathan Kelly Larch
Jonathan Kelly Larch
GAGuy Amster
Guy Amster
Background: The suitability of artificial intelligence (AI) and large language models (LLMs) to assist in curating real-world data (RWD) from electronic health records (EHR) for research holds transformative potential. Programmed death-ligand 1 (PD-L1) biomarker testing guides cancer treatment decisions, but results are hard to access because lab reports are unstructured and require clinical expertise to interpret. Additionally, results vary by cancer type, and documentation patterns have changed over time. This study explored the ability of LLMs to rapidly extract PD-L1 biomarker details from the EHR. Materials and Methods: We applied open-source LLMs (Llama-2-7B and Mistral-v0.1-7B) to extract seven biomarker details relating to PD-L1 testing from the Flatiron Health US nationwide EHR-derived database: collection/receipt/report date, cell type, percent staining, combined positive score, and staining intensity. Two approaches were used: zero-shot experiments (no fine-tuning) exploring a range of prompts and fine-tuning on manually curated answers from 500, 1000, and 1500 documents. In both cases, we validated performance using 250 human-abstracted answers across 10 cancer types. Additionally, we compared the LLM’s ability to extract PD-L1 percent staining to a deep learning model baseline trained on >10,000 examples. Results: We successfully used LLMs to extract biomarker testing details from EHR documents. Fine-tuned outputs consistently conformed to the desired RWD structure. In contrast, zero-shot outputs were frequently invalid and exhibited hallucinations. Fine-tuning performance improved with additional training examples. F1 scores ranged from 0.8 to 0.95, and date accuracy (within 15 days) ranged from 0.85 to 0.9. Fine-tuned LLMs exceeded the performance of the deep learning model baseline (ΔF1 = 0.05) despite the significant difference in training data. Conclusion: LLMs, fine-tuned with high-quality labeled data, accurately extracted complex PD-L1 test details from EHRs despite considerable variability in cancer type, documentation, and time. In contrast, zero-shot prompt extraction was not effective at the model scale examined here. Validation required access to high-quality data labeled by experts with access to the source EHR.

6 March 2023 in medRxiv
Verified
Preprint
Open access
PDF file
Blythe AdamsonBlythe Adamson
Blythe Adamson
MWMichael Waskom
Michael Waskom
ABAuriane Blarre
Auriane Blarre
JKJonathan Kelly
Jonathan Kelly
KKKonstantin Krismer
Konstantin Krismer
ACAaron Benjamin Cohen
Aaron Benjamin Cohen
Background As artificial intelligence (AI) continues to advance with breakthroughs in natural language processing (NLP) and machine learning (ML), such as the development of models like OpenAI’s ChatGPT, new opportunities are emerging for efficient curation of electronic health records (EHR) into real-world data (RWD) for evidence generation in oncology. Our objective is to describe the research and development of industry methods to promote transparency and explainability. Methods We applied NLP with ML techniques to train, validate, and test the extraction of information from unstructured documents (eg, clinician notes, radiology reports, lab reports, etc.) to output a set of structured variables required for RWD analysis. This research used a nationwide electronic health record (EHR)-derived database. Models were selected based on performance. Variables curated with an approach using ML extraction are those where the value is determined solely based on an ML model (ie, not confirmed by abstraction), which identifies key information from visit notes and documents. These models do not predict future events or infer missing information. Results We developed an approach using NLP and ML for extraction of clinically meaningful information from unstructured EHR documents and found high performance of output variables compared with variables curated by manually abstracted data. These extraction methods resulted in research-ready variables including initial cancer diagnosis with date, advanced/metastatic diagnosis with date, disease stage, histology, smoking status, surgery status with date, biomarker test results with dates, and oral treatments with dates. Conclusions NLP and ML enable the extraction of retrospective clinical data in EHR with speed and scalability to help researchers learn from the experience of every person with cancer.