Chujun Shi

Chujun Shi

Auto-generated profile, based on publicly available information

Chujun Shi

Department of Nursing, Shantou University Medical College, Shantou, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 August 2025 in JMIR Medical Education
Verified
Journal Article
Open access
YLYang Liu
Yang Liu
CSChujun Shi
Chujun Shi
LWLiping Wu
Liping Wu
XLXiule Lin
Xiule Lin
XCXiaoqin Chen
Xiaoqin Chen
WZWeishan Zhang
Weishan Zhang
Background: History-taking is crucial in medical training. However, current methods often lack consistent feedback and standardized evaluation and have limited access to standardized patient (SP) resources. Artificial intelligence (AI)–powered simulated patients offer a promising solution; however, challenges such as human-AI consistency, evaluation stability, and transparency remain underexplored in multicase clinical scenarios. Objective: This study aimed to develop and validate the AI-Powered Medical History-Taking Training and Evaluation System (AMTES), based on DeepSeek-V2.5 (DeepSeek), to assess its stability, human-AI consistency, and transparency in clinical scenarios with varying symptoms and difficulty levels. Methods: We developed AMTES, a system using multiple strategies to ensure dialog quality and automated assessment. A prospective study with 31 medical students evaluated AMTES’s performance across 3 cases of varying complexity: a simple case (cough), a moderate case (frequent urination), and a complex case (abdominal pain). To validate our design, we conducted systematic baseline comparisons to measure the incremental improvements from each level of our design approach and tested the framework’s generalizability by implementing it with an alternative large language model (LLM) Qwen-Max (Qwen AI; version 20250409), under a zero-modification condition. Results: A total of 31 students practiced with our AMTES. During the training, students generated 8606 questions across 93 history-taking sessions. AMTES achieved high dialog accuracy: 98.6% (SD 1.5%) for cough, 99.0% (SD 1.1%) for frequent urination, and 97.9% (SD 2.2%) for abdominal pain, with contextual appropriateness exceeding 99%. The system’s automated assessments demonstrated exceptional stability and high human-AI consistency, supported by transparent, evidence-based rationales. Specifically, the coefficients of variation (CV) were low across total scores (0.87%‐1.12%) and item-level scoring (0.55%‐0.73%). Total score consistency was robust, with the intraclass correlation coefficients (ICCs) exceeding 0.923 across all scenarios, showing strong agreement. The item-level consistency was remarkably high, consistently above 95%, even for complex cases like abdominal pain (95.75% consistency). In systematic baseline comparisons, the fully-processed system improved ICCs from 0.414/0.500 to 0.923/0.972 (moderate and complex cases), with all CVs ≤1.2% across the 3 cases. A zero-modification implementation of our evaluation framework with an alternative LLM (Qwen-Max) achieved near-identical performance, with the item-level consistency rates over 94.5% and ICCs exceeding 0.89. Overall, 87% of students found AMTES helpful, and 83% expressed a desire to use it again in the future. Conclusions: Our data showed that AMTES demonstrates significant educational value through its LLM-based virtual SPs, which successfully provided authentic clinical dialogs with high response accuracy and delivered consistent, transparent educational feedback. Combined with strong user approval, these findings highlight AMTES’s potential as a valuable, adaptable, and generalizable tool for medical history-taking training across various educational contexts.

26 December 2022 in BMJ Open
Verified
Journal Article
Open access
PDF file
BLBaolin Luo
Baolin Luo
ZLZebing Luo
Zebing Luo
XZXiaoyun Zhang
Xiaoyun Zhang
MXMeiwan Xu
Meiwan Xu
CSChujun Shi
Chujun Shi
Objective To investigate the risk factors of cognitive frailty in elderly patients with chronic kidney disease (CKD), and to establish an artificial neural network (ANN) model. Design A cross-sectional design. Setting Two tertiary hospitals in southern China. Participants 425 elderly patients aged ≥60 years with CKD. Methods Data were collected via questionnaire investigation, anthropometric measurements, laboratory tests and electronic medical records. The 425 samples were randomly divided into a training set, test set and validation set at a ratio of 5:3:2. Variables were screened by univariate and multivariate logistic regression analyses, then an ANN model was constructed. The accuracy, specificity, sensitivity, receiver operating characteristic (ROC) curve and area under the ROC curve (AUC) were used to evaluate the predictive power of the model. Results Barthel Index (BI) score, albumin, education level, 15-item Geriatric Depression Scale score and Social Support Rating Scale score were the factors influencing the occurrence of cognitive frailty (p<0.05). Among them, BI score was the most important factor determining cognitive frailty, with an importance index of 0.30. The accuracy, specificity and sensitivity of the ANN model were 86.36%, 88.61% and 80.65%, respectively, and the AUC of the constructed ANN model was 0.913. Conclusion The ANN model constructed in this study has good predictive ability, and can provide a reference tool for clinical nursing staff in the early prediction of cognitive frailty in a high-risk population.

24 February 2026 in JMIR Medical Education
Verified
Journal Article
Open access
YLYang Liu
Yang Liu
YZYiying Zhu
Yiying Zhu
WZWeishan Zhang
Weishan Zhang
XLXian Lu
Xian Lu
LWLiping Wu
Liping Wu
CSChujun Shi
Chujun Shi
Background: Medical history-taking is a core clinical skill; yet, traditional teaching methods face challenges. We developed an artificial intelligence–powered medical history-taking training and evaluation system (AMTES) and established its technical feasibility as an extracurricular resource. Evidence on whether such tools improve learning outcomes when voluntarily embedded in routine curricula remains limited. Objective: This study aimed to evaluate the real-world educational effectiveness of AMTES as an opt-in extracurricular tool and examine whether learning gains vary by practice patterns and baseline academic ability. Methods: We conducted a retrospective cohort study of the 2024-2025 Diagnostics course cohort (N=478) at Shantou University Medical College, China, using total population sampling. Students were categorized as AMTES users (n=205, 42.9%; ≥1 sessions) and nonusers (n=273, 57.1%) based on their voluntary extracurricular adoption of the system during the month preceding a high-stakes final practical skills examination. To address selection bias, we performed 1:1 propensity score matching via logistic regression using age, sex, and 3 previous academic scores as covariates. The average treatment effect on the treated for final examination score (0-70) was estimated with paired t tests, and robustness to unobserved confounding was assessed via Rosenbaum sensitivity analysis. Among matched users, practice patterns were identified using K-means clustering on log-derived features, with cluster differences compared using Mann-Whitney U tests. Subsequently, we explored aptitude-treatment interaction by testing the interaction between practice intensity and baseline ability using linear and logistic regression models. Results: Propensity score matching yielded 157 matched pairs (n=314) with excellent covariate balance (|standardized mean difference|<0.1). In the matched cohort, the users outperformed nonusers by 3% (average treatment effect on the treated=2.09, 95% CI 0.75-3.42; P=.002). This finding was robust to weak unmeasured confounding (Rosenbaum Γ=1.23). Among users (N=157), cluster analysis of usage logs revealed a low-intensity group (74/157, 47.1%) and a high-intensity group (83/157, 52.9%). The 2 groups reflected differences in both practice quantity and quality. However, the added efforts did not translate into higher scores (mean difference=1.6 points, 95% CI −0.5 to 3.6) or excellence probability (risk difference=7.7 percentage points, 95% CI −5.0 to 20.5). Exploratory aptitude-treatment interaction analyses suggested ability-dependent effects for excellence rate (β3=1.461; P=.04) and marginally for final score (β3=2.58; P=.07), but not for pass rate (P=.94). Conclusions: Building upon previous technical validation, this study contributes real-world effectiveness evidence by evaluating AMTES as a voluntary extracurricular supplement within an authentic, high-baseline curriculum. Unlike previous work focusing on technical feasibility or short-term controlled trials, voluntary extracurricular AMTES use was associated with modest yet meaningful improvements in summative history-taking performance. Exploratory analyses indicated that the added value of more intensive engagement may be moderated by baseline academic ability. These findings support the scalability of artificial intelligence–enabled supplementary training and inform precision-oriented instructional design.

13 April 2026 in JMIR Medical Education
Verified
Journal Article
Open access
NCNaping Chen
Naping Chen
LTLuzhen Tang
Luzhen Tang
YLYang Liu
Yang Liu
CLChangmin Lin
Changmin Lin
ZLZijian Li
Zijian Li
XLXinyu Li
Xinyu Li
Background: Effective history taking helps clinicians identify key symptoms and form accurate hypotheses. Generative artificial intelligence (GenAI)–based virtual patients (VPs) are increasingly used to simulate and practice history taking. However, there is currently no straightforward approach to effectively identify students’ clinical reasoning activities during these interactions, which limits the ability to provide instructional feedback. Objective: This study aims to develop and validate a coding scheme to identify medical students’ history-taking behaviors during interactions with GenAI-based VPs. Methods: Second-year medical students (N=210) participated in 5 history-taking cases with GenAI-based VPs, yielding 1030 dialogues. Researchers applied the systematic text condensation method to these dialogue data from cases 1 to 4 to inductively develop a coding scheme and validate coding consistency. Subsequently, the dialogue data from case 5 were used to assess the correlation between the students’ history-taking behaviors and their academic performance, including diagnostic accuracy, history-taking checklist scores, clinical knowledge test scores, and postencounter form scores. Results: A coding scheme comprising 12 behaviors across 3 dimensions—clinical reasoning behaviors, information gathering behaviors, and social interaction behaviors—was developed with high interrater reliability (κ≥0.85). The correlation analysis revealed that key clinical reasoning behaviors, such as summarizing and integrating and logical organization , showed significant positive correlations with multiple performance metrics, underscoring their importance in fostering clinical competence. In contrast, information gathering behaviors such as specifying symptoms and routine question were associated with clinical knowledge and thoroughness of history taking but were less predictive of diagnostic accuracy. Conclusions: This study developed a reliable, theory-informed coding scheme that can identify students’ questioning behaviors during history taking with GenAI-based VPs. The scheme effectively captures higher-order cognitive strategies and provides valuable insights into the development of clinical reasoning in medical students. This approach offers a scalable and efficient way to integrate real-time feedback into future medical education, fostering personalized learning and advancing competency-based assessments in clinical training.

15 May 2026 in JMIR Medical Education
Verified
Journal Article
Open access
YLYang Liu
Yang Liu
YZYiying Zhu
Yiying Zhu
CSChujun Shi
Chujun Shi
XLXian Lu
Xian Lu
LWLiping Wu
Liping Wu
WZWeishan Zhang
Weishan Zhang
Background: AI-powered virtual patient systems provide medical students with repeatable practice environments for history-taking training. However, user acceptance of such systems and the experience dimensions associated with that acceptance lack mixed methods evidence. Objective: This study aimed to (1) examine the associations of system experience and learning experience/intrinsic motivation with overall acceptance among undergraduate medical students using an AI-powered medical history-taking training and evaluation system (AMTES), (2) explore user experience patterns through open-ended questions, and (3) integrate quantitative and qualitative findings to inform system refinement and pedagogical implementation. Methods: A cross-sectional convergent mixed methods design was used. A total of 66 undergraduate medical students at a Chinese medical college completed a postuse questionnaire after AMTES training. The primary outcome was an overall acceptance composite combining use intention, recommendation intention, and overall satisfaction. Associations with system experience and learning experience/intrinsic motivation were examined using linear regression with heteroscedasticity-consistent SEs (type 3) and bootstrap CIs. Sensitivity analyses included covariate adjustment, single-outcome models, a fractional logit model, and content-overlap sensitivity checks for the system-experience composite. Open-ended responses were analyzed using codebook-oriented thematic analysis and integrated with quantitative findings through a joint display. Results: Both system experience and learning experience/intrinsic motivation were positively associated with overall acceptance (standardized β=.526; P<.001 and standardized β=.377; P=.002, respectively; R2=0.662). Sensitivity analyses supported the robustness of both positive associations. Qualitative analysis showed that the most frequently nominated benefits clustered within the “Practice Accessibility and Feedback Support” theme, particularly self-directed practice (28/66, 42.4%) and immediate feedback (25/66, 37.9%). Refinement priorities clustered around 4 dialogue-quality and assessment dimensions, including semantic understanding, contextual consistency, conversational naturalness, and scoring logic, each cited by 21%-35% of respondents. The mixed methods joint display indicated contextual (same-source) alignment on dialogue and scoring concerns and suggested that perceived scoring-feedback discrepancies may be associated with lower student trust in the feedback function. This cross-dimensional pattern was not apparent from the quantitative model alone. Conclusions: In this exploratory cohort of undergraduate medical students, both system interaction quality and perceived learning value were positively associated with overall acceptance of AMTES, with system interaction quality showing the stronger association within the study’s measurement specification. Dialogue coherence, semantic understanding, and scoring-feedback alignment emerged as the most frequently nominated refinement priorities and are plausible candidate targets for improving acceptance-related perceptions. This study emphasizes implementation-level acceptance rather than solely technical reliability or educational effectiveness. Interaction-quality problems may be associated with less favorable acceptance even when learning value is recognized. These findings may inform system refinement priorities and the curricular integration of AI-powered history-taking training systems, while larger multicenter and longitudinal studies are needed to examine whether improvements in interaction quality translate into gains in learner acceptance and downstream training outcomes.

2 December 2024 in Current Psychology
Verified
Journal Article
Closed access
PDF file
XZXiaoyun Zhang
Xiaoyun Zhang
YCYu Chen
Yu Chen
QCQiongshan Chen
Qiongshan Chen
NZNanlan Zheng
Nanlan Zheng
CSChujun Shi
Chujun Shi