Zeyu Yu

Mr. Zeyu Yu


Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 May 2025 in Applied Sciences
Verified
Journal Article
Open access
PDF file
YWYonggu Wang
Yonggu Wang
ZYZeyu Yu
Zeyu Yu
ZWZIhan Wang
ZIhan Wang
ZYZengyi Yu
Zengyi Yu
JWJue Wang
Jue Wang
The question generation system (QGS) for information technology (IT) education, designed to create, evaluate, and improve Multiple-Choice Questions (MCQs) using knowledge graphs (KGs) and large language models (LLMs), encounters three major needs: ensuring the generation of contextually relevant and accurate distractors, enhancing the diversity of generated questions, and balancing the higher-order thinking of questions to match various learning levels. To address these needs, we proposed a multi-agent system named Multi-Examiner, which integrates KGs, domain-specific search tools, and local knowledge bases, categorized according to Bloom’s taxonomy, to enhance the contextual relevance, diversity, and higher-order thinking of automatically generated information technology MCQs. Our methodology employed a mixed-methods approach combining system development with experimental evaluation. We first constructed a specialized architecture combining knowledge graphs with LLMs, then implemented a comparative study generating questions across six knowledge points from K-12 Computer Science Standard. We designed a multidimensional evaluation rubric to assess the semantic coherence, answer correctness, question validity, distractor relevance, question diversity, and higher-order thinking, and conducted a statistical analysis of ratings provided by 30 high school IT teachers. Results showed statistically significant improvements (p < 0.01) with Multi-Examiner outperforming GPT-4 by an average of 0.87 points (on a 5-point scale) for evaluation-level questions and 1.12 points for creation-level questions. The results demonstrated that: (i) overall, questions generated by the Multi-Examiner system outperformed those generated by GPT-4 across all dimensions and closely matched the quality of human-crafted questions in several dimensions; (ii) domain-specific search tools significantly enhanced the diversity of questions generated by Multi-Examiner; and (iii) GPT-4 generated better questions for knowledge points at the “remembering” and “understanding” levels, while Multi-Examiner significantly improved the higher-order thinking of questions for the “evaluating” and “creating” levels. This study contributes to the growing body of research on AI-supported educational assessment by demonstrating how specialized knowledge structures can enhance automated generation of higher-order thinking questions beyond what general-purpose language models can achieve.

14 March 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
YWYonggu Wang
Yonggu Wang
ZYZeyu Yu
Zeyu Yu
ZYZengyi Yu
Zengyi Yu
ZWZIhan Wang
ZIhan Wang
JWJue Wang
Jue Wang
The Question Generation System (QGS) for Information Technology (IT) education, designed to create, evaluate, and improve Multiple-Choice Questions (MCQs) using Knowledge graphs (KGs) and Large Language Models (LLMs), encounters three major needs: ensuring the generation of contextually relevant and accurate distractors, enhancing the diversity of generated questions, and balancing the higher-order thinking of questions to match various learning levels. To address these needs, we proposed a multi-agent system named Multi-Examiner, which integrates knowledge graphs, domain-specific search tools, and local knowledge bases, categorized according to Bloom’s taxonomy, to enhance the contextual relevance, diversity, and higher-order thinking of automatically generated information technology multiple-choice questions. We designed a multidimensional evaluation rubric to assess the semantic coherence, answer correctness, question validity, distractor relevance, question diversity, and higher-order thinking, and applied it to questions generated for six knowledge points from the second chapter of the "Information Systems and Society" textbook using both the Multi-Examiner system and GPT-4, alongside real exam questions, evaluated by 30 high school IT teachers. The results demonstrated that: (i) overall, questions generated by the Multi-Examiner system outperformed those generated by GPT-4 across all dimensions and closely matched the quality of human-crafted questions in several dimensions; (ii) domain-specific search tools significantly enhanced the diversity of questions generated by Multi-Examiner; (iii) GPT-4 generated better questions for knowledge points at the "remembering" and "understanding" levels, while Multi-Examiner significantly improved the higher-order thinking of questions for "evaluating" and "creating" levels. This study highlights the potential of multi-agent systems in advancing question generation.