Artificial intelligence (AI) driven chatbots provide instant feedback to support learning. Yet, the impacts of different feedback types on behavior and brain activation remain underexplored. We investigated how metacognitive, affective, and neutral feedback from an educational chatbot affected learning outcomes and brain activity using functional near-infrared spectroscopy. Students receiving metacognitive feedback showed higher transfer scores, greater metacognitive sensitivity, and increased brain activation in the frontopolar area and middle temporal gyrus compared to other feedback types. Such activation correlated with metacognitive sensitivity. Students receiving affective feedback showed better retention scores than those receiving neutral feedback, along with higher activation in the supramarginal gyrus. Students receiving neutral feedback exhibited higher activation in the dorsolateral prefrontal cortex than other feedback types. The machine learning model identified key brain regions that predicted transfer scores. These findings underscore the potential of diverse feedback types in enhancing learning via human-chatbot interaction, and provide neurophysiological signatures.
23 September 2024 in Education and Information Technologies
Educational dialogue classification is a critical task for analyzing classroom interactions and fostering effective teaching strategies. However, the scarcity of annotated data and the high cost of manual labeling pose significant challenges, especially in low-resource educational contexts. This article presents the EduDCM framework for the first time, offering an original approach to addressing these challenges. EduDCM innovatively integrates distant supervision with the capabilities of Large Language Models (LLMs) to automate the construction of high-quality educational dialogue classification datasets. EduDCM reduces the noise typically associated with distant supervision by leveraging LLMs for context-aware label generation and incorporating heuristic alignment techniques. To validate the framework, we constructed the EduTalk dataset, encompassing diverse classroom dialogues labeled with pedagogical categories. Extensive experiments on EduTalk and publicly available datasets, combined with expert evaluations, confirm the superior quality of EduDCM-generated datasets. Models trained on EduDCM data achieved a performance comparable to that of manually annotated datasets. Expert evaluations using a 5-point Likert scale show that EduDCM outperforms Template-Based Generation and Few-Shot GPT in terms of annotation accuracy, category coverage, and consistency. These findings emphasize EduDCM’s novelty and its effectiveness in generating high-quality, scalable datasets for low-resource educational NLP tasks, thus reducing manual annotation efforts.
Knowledge tracing is a crucial task in personalized learning that models student mastery based on historical data to predict future performance. Currently, deep learning models in knowledge tracing predominantly use one-hot encodings of question, knowledge, and student IDs, showing promising results. However, they face a significant limitation: a vocabulary gap that impedes the processing of new IDs not seen during training. To address this, our paper introduces a novel method that incorporates aggregated features, termed ‘side information’, that captures essential attributes such as student ability, knowledge mastery, and question difficulty. Our approach utilizes side information to bridge the vocabulary gap caused by ID-based one-hot encoding in traditional models. This enables the model, once trained on one dataset, to generalize and make predictions on new datasets with unfamiliar students, knowledge, or questions without the need for retraining. This innovation effectively bridges the vocabulary gap, reduces the dependency on specific data representations, and improves the overall performance of the model. Experimental evaluations on five distinct datasets show that our proposed model consistently outperforms baseline models, using fewer parameters and demonstrating seamless adaptability to new contexts. Additionally, ablation studies highlight that including side information, especially regarding students and questions, significantly improves knowledge tracing effectiveness. In summary, our approach not only resolves the vocabulary gap challenge but also offers a more robust and superior solution across varied datasets.