Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.

Information about the paper titled "DEVELOPING A DIGITAL COMPETENCE ASSESSMENT AND ACCREDITATION PLATFORM FOR DIGITAL PROFILES" at IATED Digital Library

Background: When measuring complex cognitive constructs, it is crucial to correctly design the evaluation items in order to trigger the intended knowledge and skills. Furthermore, assessing the validity of an assessment requires considering not only the content of the evaluation tasks, but also how examinees perform by engaging construct-relevant response processes. Objectives: We used eye-tracking techniques to examine item response processes in the assessment of digital competence. The eye-tracking observations helped to fill an ‘explanatory gap’ by providing data on the variation in response processes that cannot be captured by other common sources. Method: Specifically, we used eye movement data to validate the inferences made between claimed and observed behavior. This allowed us to interpret how participants processed the information in the items in terms of Area Of Interest (their size, placement, and order). Results and Conclusions: The gaze data provide detailed information about response strategies at the item level, profiling the examinees according to their engagement, response processes and performance/success rate. The presented evidence confirms that the response patterns of the participants who responded well do not represent an alternative to the interpretation of the results that would undermine the assessment criteria. Takeaways: Gaze-based evidence has great potential to provide complementary data about the response processes performed by examinees, thereby contributing to the validity argument.

Soft skills remain fundamental for employability and sustainable human development in an increasingly technology-driven society. These interpersonal and cognitive competencies—such as communication, adaptability, and critical thinking—represent uniquely human capabilities that current Artificial Intelligence (AI) systems cannot replicate. However, assessing and developing these skills consistently remains a challenge due to the lack of standardized evaluation frameworks. This study explores the potential of commercial video games as engaging environments for soft skills enhancement and introduces an AI-based assessment methodology to quantify such improvement. Using player data collected from the Steam platform, we designed and validated an AI model based on Gradient Boosting Regressor (GBR) to estimate participants’ soft skill progression. The model achieved high predictive performance (R² ≈ 0.9; MAE/RMSE ≈ 1), demonstrating strong alignment between gameplay behavior and soft skill improvement. The results highlight that video game-based data analysis can provide a reliable, non-intrusive alternative to traditional testing methods, reducing test-related anxiety while maintaining assessment validity. This approach supports the integration of video games into educational and professional training frameworks as a scalable and data-driven tool for soft skills development.