This article provides a comprehensive overview of artificial intelligence (AI) for serious games. Reporting about the work of a European flagship project on serious game technologies, it presents a set of advanced game AI components that enable pedagogical affordances and that can be easily reused across a wide diversity of game engines and game platforms. Serious game AI functionalities include player modelling (real-time facial emotion recognition, automated difficulty adaptation, stealth assessment), natural language processing (sentiment analysis and essay scoring on free texts), and believable non-playing characters (emotional and socio-cultural, non-verbal bodily motion, and lip-synchronised speech), respectively. The reuse of these components enables game developers to develop high quality serious games at reduced costs and in shorter periods of time. All these components are open source software and can be freely downloaded from the newly launched portal at gamecomponents.eu. The components come with detailed installation manuals and tutorial videos. All components have been applied and validated in serious games that were tested with real end-users.
This paper presents a methodology for applying automated essay scoring in educational settings. The methodology was tested and validated on a dataset of 173 reports (in Dutch language) that students have created in an applied game on environmental policy. Natural Language Processing technologies from the ReaderBench framework were used to generate an extensive set of textual complexity indices for each of the reports. Afterwards, different machine learning algorithms were used to predict the scores. By combining binary classification (pass or fail) and a probabilistic model for precision, a trade-off can be made between validity of automated score prediction (precision) and the reduction of teacher workload required for manual assessment. It was found from the sample that substantial workload reduction can be achieved, while preserving high precision: allowing for a precision of 95% or higher would already reduce the teacher's workload to 74%; lowering precision to 80% produces a workload reduction of 50%.
The unprecedented scale of disinformation on the Internet for more than a decade represents a serious challenge for democratic societies. When this process is focused on a well-established subject such as climate change, it can subvert measures and policies that various governmental bodies have taken to mitigate the phenomenon. It is therefore essential to effectively identify and counteract fake news on climate change. To do this, our main contribution represents a novel dataset with more than 2300 articles written in French, gathered using web scraping from all types of media dealing with climate change. Manual labeling was performed by two annotators with three classes: “fake”, “biased”, and “true”. Machine Learning models ranging from bag-of-words representations used by an SVM to Transformer-based architectures built on top of CamemBERT were built to automatically classify the articles. Our results, with an F1-score of 84.75% using the BERT-based model at the article level coupled with hand-crafted features specifically tailored for this task, represent a strong baseline. At the same time, we highlight perceptual properties as text sequences (i.e., fake, biased, and irrelevant text fragments) at the sentence level, with a macro F1 of 45.01% and a micro F1 of 78.11%. Based on these results, our proposed method facilitates the identification of fake news, and thus contributes to better education of the public.
Keystroke authentication is a well-established biometric technique that has gained significant attention due to its non-intrusive and continuous characteristics. The method analyzes the unique typing patterns of individuals to verify their identity while interacting with the keyboard, both virtual and hardware. Current deep-learning approaches like TypeNet and TypeFormer focus on generating biometric signatures as embeddings for the entire typing sequence. The authentication process is defined using the Euclidean distances between the new typing embedding and the saved biometric signatures. This paper introduces a novel approach called DoubleStrokeNet for authenticating users through keystroke analysis using bigram embeddings. Unlike conventional methods, our model targets the temporal features of bigrams to generate user embeddings. This is achieved using a Transformer-based neural network that distinguishes between different bigrams. Furthermore, we employ self-supervised learning techniques to compute embeddings for both bigrams and users. By harnessing the power of the Transformer’s attention mechanism, the DoubleStrokeNet approach represents a significant departure from existing methods. It allows for a more precise and accurate assessment of user authenticity, specifically emphasizing the temporal characteristics and latent representations of bigrams in deriving user embeddings. Our experiments were conducted using the Aalto University keystrokes datasets, which include 136 million keystrokes from 168,000 subjects using physical keyboards and 63 million keystrokes acquired on mobile devices from 60,000 subjects. The DoubleStrokeNet outperforms the TypeNet-based authentication system using 10 enrollment typing sequences, achieving Equal Error Rate (EER) values of 0.75% and 2.35% for physical and touchscreen keyboards, respectively.
1 April 2024 in International Journal of Artificial Intelligence in Education
Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline integrated into the ReaderBench platform designed to simplify the process of training AES models by automating both feature extraction and architecture tuning for any multilingual dataset uploaded by the user. The dataset must contain a list of texts, each with potentially multiple annotations, either scores or labels. The platform includes traditional ML models relying on linguistic features and a hybrid approach combining Transformer-based architectures with the previous features. Our method was evaluated on three publicly available datasets in three different languages (English, Portuguese, and French) and compared with the best currently published results on these datasets. Our automated approach achieved comparable results to state-of-the-art models on two datasets, while it obtained the best performance on the third corpus in Portuguese.