This paper explores the potential of Large Language Models (LLMs) in automating the grading and feedback generation of short-answer responses in Arabic. Arabic poses unique challenges due to its linguistic complexity and the relative scarcity of well-developed Natural Language Processing (NLP) resources compared to languages such as English and Chinese. The study evaluates both proprietary models (GPT-4) and open-source models (Llama 3-8B, Llama 3-70B, and DeepSeek-V3) using the Environmental Science Corpus—a custom-designed dataset tailored for Arabic short-answer assessment. Two core tasks are addressed: grading and feedback generation. In the grading task, DeepSeek-V3 achieved the best performance, with a Quadratic Weighted Kappa (QWK) score of 0.8273, a Pearson correlation of 86.09%, and a Root Mean Squared Error (RMSE) of 0.76, indicating near-perfect agreement with human evaluators. GPT-4 ranked second, followed by Llama 3-70B, while Llama 3-8B was the lowest-performing model. In feedback generation, DeepSeek-V3 again led the performance with a human evaluation score of 79.61% for generating accurate and constructive feedback. GPT-4 ranked second, followed by Llama 3 models. Statistical analysis using the Wilcoxon test revealed significant performance differences among all models (p < 0.05), indicating that each LLM offers unique capabilities in handling Arabic short-answer grading. Overall, the results underscore the effectiveness of LLMs in Arabic educational assessment, highlighting the critical role of prompt engineering in enhancing model performance. The study demonstrates that LLMs can not only grade student responses with high accuracy but also generate meaningful feedback, thereby supporting the development of more effective automated learning tools. Practical recommendations and best practices are presented to help educators and developers optimize the use of LLMs in Arabic-language educational settings, laying the groundwork for future advancements in Arabic NLP.

This study introduces an intelligent scoring approach that leverages natural language processing and transformer-based models to evaluate student responses across various academic subjects. Given the linguistic complexity and variability of Arabic short-answer questions, the research proposes a novel grading method that moves beyond traditional techniques. Using the Cairo University Dataset a widely recognized benchmark focused on environmental science the study explores different preprocessing strategies and applies multiple transformer models. These models are integrated into a custom regression-based neural network designed specifically for Arabic short-answer grading. The proposed system achieves a Pearson correlation of 92.34%, surpassing the current state-of-the-art on the Cairo University Dataset. To evaluate generalizability, the model was also tested on the Arabic Short Answer Grading dataset, achieving an 80% Pearson correlation and outperforming existing benchmarks. These results demonstrate the approach’s strong potential for educational applications, offering a scalable and fair grading solution that reduces teacher workload while maintaining assessment accuracy.