The recent advancements in large language models (LLMs) have brought significant changes to the field of education, particularly in the generation and evaluation of feedback. LLMs are transforming education by streamlining tasks like content creation, feedback generation, and assessment, reducing teachers’ workload and improving online education efficiency. This study aimed to verify the consistency and reliability of LLMs as evaluators by conducting automated evaluations using various LLMs based on five educational evaluation criteria. The analysis revealed that while LLMs were capable of performing consistent evaluations under certain conditions, a lack of consistency was observed both among evaluators and across models for other criteria. Notably, low agreement among human evaluators correlated with reduced reliability in LLM evaluations. Furthermore, variations in evaluation results were influenced by factors such as prompt strategies and model architecture, highlighting the complexity of achieving reliable assessments using LLMs. These findings suggest that while LLMs have the potential to transform educational systems, careful selection and combination of models are essential to improve their consistency and align their performance with human evaluators in educational settings.

A major challenge in education is to provide students with a personalized learning experience. This study aims to address this by developing a dialogue-based intelligent tutoring system (ITS) that imitates human expert tutors. The ITS asks questions, assesses student answers, provides hints, and even chats to encourage student engagement. We constructed the Dialogue-based Reading Comprehension Tutoring (DIRECT) dataset to simulate real-world pedagogical scenarios with the assessment labels and key sentences to support tutoring. The DIRECT dataset is based on RACE, which is a large-scale English reading comprehension dataset. In addition, we propose a neural pipeline approach to model the tutoring tasks and conduct a comprehensive analysis on the results, including a human evaluation. The results show that our model performs well in generating questions, assessing answers, and chatting, showing high potential although some challenges remain. The proposed model provides a good basis for further development of dialogue-based ITSs. The dataset will be publicly available after the publication.

To translate in real time, a simultaneous translation system should determine when to stop reading source tokens and generate target tokens corresponding to a partial source sentence read up to that ...

Junyeong Park, Seogyeong Jeong, Seyoung Song, Yohan Lee, Alice Oh. Proceedings of the 3rd Workshop on Cross-Cultural Considerations in NLP (C3NLP 2025). 2025.

Considerable research has been conducted to obtain translations that reflect contextual information in documents and simultaneous interpretations. Most of the existing studies use concatenation data which merge previous and current sentences for training translation models. Although this corpus improves the performance of the model, ignoring the contextual correlation between the sentences can disturb translation performance. In this study, we introduce a simple and effective method to capture the contextual correlation of the sentence at the document level of the current sentence, thereby learning an effective contextual representation. In addition, the proposed model structure is applied to a separate residual connection network to minimize the loss of the beneficial influence of incorporating the context. The experimental results show that our methods improve the translation performance in comparison with the state-of-the-art baseline of the Transformer in various translation tasks and two benchmark machine translation tasks.

Hyuk Namgoong, Jeesu Jung, Hyeonseok Kang, Yohan Lee, Sangkeun Jung. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.