Yu Chen

Yu Chen

Auto-generated profile, based on publicly available information

Yu Chen

Nanjing Medical University, Nanjing, Jiangsu, China

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 October 2025 in JMIR Preprints
Verified
Preprint
Open access
YXYifan Xu
Yifan Xu
JMJianhao Ma
Jianhao Ma
YHYujia Hu
Yujia Hu
YLYixue Liu
Yixue Liu
YCYu Chen
Yu Chen
RHRuochen Huang
Ruochen Huang
BACKGROUND Online health communities (OHCs) are important channels for families of children with autism spectrum disorder to obtain health information and psychosocial support. Differences between an open forum platform and the physician-patient consultation platforms may shape caregiver decisions, yet comparative evidence from China remains limited. A large language model (LLM) provides a scalable approach for systematic content annotation in large OHC datasets. OBJECTIVE This study proposes and validates a standardized LLM-assisted annotation framework under a unified classification schema and compares topic distributions and poster identities across an open forum platform (Baidu Tieba) and physician-patient consultation platforms (Chunyu Doctor and Haodf). METHODS We implemented an LLM-assisted annotation framework. A unified taxonomy of topics and poster identities was first developed through human open coding. Poster identities in Baidu Tieba were annotated through a double-blind manual procedure. For topic classification, interannotator and human-LLM agreement were evaluated on a manually labeled subset to benchmark models of varying sizes. The best-performing LLM was selected for full-dataset topic annotation, followed by statistical and cross-platform analysis. RESULTS When metrics were arithmetically averaged across all annotation tasks, the best-performing LLM achieved agreement levels comparable to human annotation (accuracy=79.18%, SD 0.20%; κ=0.736, SD 0.003; F1-score=0.727, SD 0.006), approaching interannotator agreement (accuracy=81.65%; κ=0.767; F1-score=0.758), demonstrating strong stability and scalability. Full-dataset analysis yielded 3 main findings. First, model performance increased with parameter scale but plateaued beyond 14B, indicating diminishing marginal returns from further scaling. Second, clear cross-platform differences in poster identity were observed: the open forum platform was dominated by family members of patients (caregivers: 2377/3516, 67.61%), with substantial participation from commercial rehabilitation practitioners (commercial posters: 427/3516, 12.14%), resulting in a more heterogeneous participation structure. Third, topic distributions reflected both shared high demand for resource-related information and differentiated help-seeking pathways: both platform types demonstrated consistently high demand for resource recommendation and evaluation; the open forum platform was primarily characterized by diagnosis-related discussions (1183/7535, 15.70%), whereas the physician-patient consultation platforms were centered on intervention-related consultation (2864/7687, 37.26%). CONCLUSIONS The LLM-assisted annotation framework proposed in this study enables reliable large-scale annotation of OHC data while maintaining high human-LLM agreement and operational stability. Midsized models (eg, 14B) demonstrated favorable cost-performance efficiency. The findings reveal 2 key aspects: the open forum platform exhibits a complex participation structure, and the influence of commercially affiliated actors should not be overlooked; users on both platform types show sustained demand for resource-related information but follow different help-seeking pathways, emphasizing diagnostic exploration and professional intervention, respectively. These results suggest that platform structure and governance mechanisms may shape caregivers’ information access and decision-making. The framework provides a transparent, reproducible, and cost-effective approach for OHC research. All data were deidentified and handled in accordance with relevant platform policies and ethical standards. CLINICALTRIAL

10 July 2026 in Journal of Medical Internet Research
Verified
Journal Article
Open access
YXYifan Xu
Yifan Xu
JMJianhao Ma
Jianhao Ma
YHYujia Hu
Yujia Hu
YLYixue Liu
Yixue Liu
YCYu Chen
Yu Chen
RHRuochen Huang
Ruochen Huang
Background: Online health communities (OHCs) are important channels for families of children with autism spectrum disorder to obtain health information and psychosocial support. Differences between an open forum platform and the physician-patient consultation platforms may shape caregiver decisions, yet comparative evidence from China remains limited. A large language model (LLM) provides a scalable approach for systematic content annotation in large OHC datasets. Objective: This study proposes and validates a standardized LLM-assisted annotation framework under a unified classification schema and compares topic distributions and poster identities across an open forum platform (Baidu Tieba) and physician-patient consultation platforms (Chunyu Doctor and Haodf). Methods: We implemented an LLM-assisted annotation framework. A unified taxonomy of topics and poster identities was first developed through human open coding. Poster identities in Baidu Tieba were annotated through a double-blind manual procedure. For topic classification, interannotator and human-LLM agreement were evaluated on a manually labeled subset to benchmark models of varying sizes. The best-performing LLM was selected for full-dataset topic annotation, followed by statistical and cross-platform analysis. Results: When metrics were arithmetically averaged across all annotation tasks, the best-performing LLM achieved agreement levels comparable to human annotation (accuracy=79.18%, SD 0.20%; κ=0.736, SD 0.003; F1-score=0.727, SD 0.006), approaching interannotator agreement (accuracy=81.65%; κ=0.767; F1-score=0.758), demonstrating strong stability and scalability. Full-dataset analysis yielded 3 main findings. First, model performance increased with parameter scale but plateaued beyond 14B, indicating diminishing marginal returns from further scaling. Second, clear cross-platform differences in poster identity were observed: the open forum platform was dominated by family members of patients (caregivers: 2377/3516, 67.61%), with substantial participation from commercial rehabilitation practitioners (commercial posters: 427/3516, 12.14%), resulting in a more heterogeneous participation structure. Third, topic distributions reflected both shared high demand for resource-related information and differentiated help-seeking pathways: both platform types demonstrated consistently high demand for resource recommendation and evaluation; the open forum platform was primarily characterized by diagnosis-related discussions (1183/7535, 15.70%), whereas the physician-patient consultation platforms were centered on intervention-related consultation (2864/7687, 37.26%). Conclusions: The LLM-assisted annotation framework proposed in this study enables reliable large-scale annotation of OHC data while maintaining high human-LLM agreement and operational stability. Midsized models (eg, 14B) demonstrated favorable cost-performance efficiency. The findings reveal 2 key aspects: the open forum platform exhibits a complex participation structure, and the influence of commercially affiliated actors should not be overlooked; users on both platform types show sustained demand for resource-related information but follow different help-seeking pathways, emphasizing diagnostic exploration and professional intervention, respectively. These results suggest that platform structure and governance mechanisms may shape caregivers’ information access and decision-making. The framework provides a transparent, reproducible, and cost-effective approach for OHC research. All data were deidentified and handled in accordance with relevant platform policies and ethical standards.