9 May 2024 in IEEE Access
Artificial intelligence technology is becoming increasingly essential to education. The outbreak of COVID-19 in recent years has led many schools to launch online education. Automated online assessments have become a hot topic of interest, and an increasing number of researchers are studying Automated Essay Scoring (AES). This work seeks to summarise the characteristics of current AES systems used in English writing assessment, identify their strengths and weaknesses, and finally, analyse the limits of recent studies and research trends. Search strings were used to retrieve papers on AES systems from 2018 to 2023 from four databases, 104 of which were chosen to be potential to address the posed research aims after study selection and quality evaluation. It is concluded that the existing AES systems, although achieving good results in terms of accuracy in specific contexts, are unable to meet the needs of teachers and students in real teaching scenarios. The improvements of these systems relate to the scalability of the system for assessing different topics or styles of the essays, the accuracy of the model’s predicted scores, as well as the reliability of outcomes: improving the robustness of AES models with some adversarial inputs, the richness of AES system functionality, and the development of AES assist tools.
21 July 2025 in International Journal of Modern Physics C
15 March 2022 in Information Research
Introduction. To reduce cost of the evaluation of information retrieval systems, this study proposes a method that employs deep learning to predict the precision evaluation metric. It also aims to show why some of existing evaluation metrics correlate with each other while considering the varying distributions of relevance assessments. It aims to ensure reproducibility of all the presented experiments. Method. Using data from several test collections of the Text REetrieval Conference (TREC) we show why some evaluation metrics correlate with each other, through mathematical intuitions. In addition, regression models were used to investigate how the predictions of the evaluation metrics are affected by queries or topics with variations of relevance assessments. Lastly, the proposed prediction method employs deep learning. Analysis. We use coefficient of determination, Kendall's tau, Spearman and Pearson correlations. Results. This study showed that the proposed method performed better predictions than other recently proposed methods in retrieval research. It also showed why the correlation exists between precision and rank biased precision metrics, and why recall and average precision metrics have reduced correlation when the cut-off depth increases. Conclusions. The proposed method and the justifications for the correlations between some pairs of retrieval metrics will be valuable to researchers for the predictions of the evaluation metrics of information retrieval systems.
19 February 2025 in IEEE Access
Presents corrections to the paper, (Corrections to “A Systematic Literature Review: Are Automated Essay Scoring Systems Competent in Real-Life Education Scenarios?”).