Cindy Lemon

Cindy Lemon

Auto-generated profile, based on publicly available information

Cindy Lemon

Health Outcomes and Policy Research Program, University of Tennessee Health Science Center, Memphis, Tennessee, USA

Research Focus & Metrics

Academic Career Length

1Year (2026-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications

24 April 2026 in JMIR Preprints
Verified
Preprint
Open access
LCLokesh Chinthala
Lokesh Chinthala
ASArash Shaban-Nejad
Arash Shaban-Nejad
HXHua Xu
Hua Xu
CLCindy Lemon
Cindy Lemon
GFGregory Farage
Gregory Farage
CMCharisse Madlock-Brown
Charisse Madlock-Brown
BACKGROUND Recent advancements in large language models (LLMs) have improved the identification of social isolation from clinical narratives, which vary widely in linguistic patterns and documentation practices. However, LLMs fine-tuned on a single dataset often show reduced performance when applied to different healthcare settings or clinical note types. Rigorous evaluation of cross-site generalizability and fairness is therefore essential to ensure accurate and equitable detection of social isolation across diverse populations and clinical contexts. OBJECTIVE This study aimed to evaluate a span-level fine-tuned FLAN-T5-Large model for extracting ‘social isolation’ indicators from unstructured clinical text and to assess its generalizability and fairness across diverse populations and healthcare data sources. METHODS A total of 2,967 unique annotated spans from 9,578 clinical notes across three healthcare systems were used to fine-tune an FLAN-T5-Large model using a contextualized span classification framework. A Gemma‑2-2B model was evaluated in a sensitivity analysis to assess architecture‑related performance differences. Performance was assessed using precision, recall, and macro F1. Fairness was evaluated across demographic variables, social vulnerability strata, and note types using statistical parity difference (SPD) and equal opportunity difference (EOD). RESULTS Incorporating contextual windows around annotated spans improved macro-F1 from 0.90 to 0.94 during validation. In full note evaluation across 900 manually reviewed notes, FLAN-T5-Large achieved high recall for social isolation (0.94 – 0.98) and macro F1 values ranging from 0.69 to 0.81 across sites. Fairness analysis showed generally consistent performance across age, gender, race, and social vulnerability groups, with equitable sensitivity (EOD 0.02 – 0.04) and moderate variation in positive prediction rates (SPD). Performance variability was strongly associated with documentation type, with note type driving substantially greater variability in both performance and fairness metrics than patient demographic factors. CONCLUSIONS Fine-tuned FLAN‑T5-Large demonstrated strong capability in detecting ‘social isolation’ from clinical narratives, while maintaining sensitivity parity across subgroups. The observed heterogeneity was largely driven by documentation context rather than by patient characteristics, highlighting the importance of note type-aware evaluation in clinical NLP. These findings support the use of instruction‑tuned LLMs for equitable extraction of social context information from EHR text.

7 July 2026 in medRxiv
Verified
Preprint
Open access
PDF file
LCLokesh Chinthala
Lokesh Chinthala
CLCindy K. Lemon
Cindy K. Lemon
ASArash Shaban-Nejad
Arash Shaban-Nejad
GFGregory Farage
Gregory Farage
RDRobert Davis
Robert Davis
CMCharisse Madlock-Brown
Charisse Madlock-Brown
Background: Social isolation and social support are clinically important but inconsistently represented in structured electronic health record data. Their identification from clinical narratives is complicated by nuanced, context-dependent language and by variation in how social connectedness is documented across clinical settings, patient populations, and geographic locations. Models intended for health care use must therefore distinguish social isolation from documented support, negation, ambiguous social circumstances, and non-social clinical uses of similar terminology while remaining robust to differences in documentation practices. Objective: This study aimed to develop and evaluate locally deployable, fine-tuned language models for context-aware classification of social isolation, social support, and contextually similar nonrelevant clinical text across multiple health systems. Materials and Methods: We conducted a retrospective study using electronic health record data from 326,847 adults aged 50 years or older who received care at 3 Tennessee health systems from 2020 to 2023. Candidate clinical text was identified using social-environment ICD-10-CM Z codes and an established social isolation lexicon. Human annotators classified 9,748 spans from 9,578 clinical notes contributed by 6,881 patients as social isolation, social support, or no social isolation. The annotated corpus included social isolation and support concepts including negations, infection-control and microbiology terminology, institutional living arrangements, and other contextually ambiguous examples. Uncertainty-based active learning was used to prioritize informative spans for annotation. FLAN-T5-Large, BERT, RoBERTa, and Gemma-2-2B were fine-tuned for 3-class span classification. Full fine-tuning and parameter-efficient fine-tuning using low-rank adaptation approaches were evaluated. Performance was assessed using site-based internal-external cross-validation, with macro-F1 as the primary metric. Results: The annotated corpus included 2,318 social isolation spans, 638 social support spans, and 6,792 spans classified as no social isolation. Fully fine-tuned FLAN-T5-Large achieved the highest and most consistent performance, with a mean macro-F1 of 0.92 (SD 0.04). Class-specific F1 scores were 0.91 (SD 0.03) for social isolation, 0.90 (SD 0.04) for social support, and 0.94 (SD 0.05) for no social isolation. Gemma-2-2B achieved a macro-F1 of 0.89 (SD 0.10), compared with 0.80 (SD 0.21) for RoBERTa and 0.77 (SD 0.17) for BERT. Full fine-tuning also outperformed parameter-efficient fine-tuning in the corresponding held-out evaluation. Remaining errors primarily involved patients who lived alone but received assistance, the presence of family members without meaningful support, grief-related language, and non-social clinical uses of isolation terminology. Conclusions: Fine-tuned language models can distinguish social isolation, social support, and contextually similar but unrelated language across heterogeneous clinical settings. FLAN-T5-Large provided the strongest and most stable performance, including for the less frequent social support class. The 3-class annotation framework, inclusion of difficult negative examples, and multisite evaluation offer a practical approach for converting nuanced social-connectedness documentation into structured data. Further work should evaluate end-to-end identification in unselected clinical notes and prospective integration into clinical and population health workflows.