A common approach for measuring the effectiveness of an education system or a school is the estimation of the impact that school interventions have on students’ academic performance. However, the latest trends aim to extend the focus beyond students’ acquisition of knowledge and skills, and to consider aspects such as well-being in the academic context. For this reason, the 2015 edition of the international assessment system Programme for International Student Assessment (PISA) incorporated a new tool aimed at evaluating the socio-emotional variables related to the well-being of students. It is based on a definition focused on the five dimensions proposed in the PISA theoretical framework: cognitive, psychological, social, physical, and material. The main purpose of this study is to identify the well-being components that significantly affect student academic performance and to estimate the magnitude of school effects on the well-being of students in OECD countries, the school effect being understood as the ability of schools to increase subjective student well-being. To achieve this goal, we analyzed the responses of 248,620 students from 35 OECD countries to PISA 2015 questionnaires. Specifically, we considered non-cognitive variables in the questionnaires and student performance in science. The results indicated that the cognitive well-being dimension, composed of enjoyment of science, self-efficacy, and instrumental motivation, as well as test anxiety all had a consistent relationship with student performance across countries. In addition, the school effect, estimated through a two-level hierarchical linear model, in terms of student well-being was systematically low. While the school effect accounted for approximately 25% of the variance in the results for the cognitive dimension, only 5–9% of variance in well-being indicators was attributable to it. This suggests that the influence of school on student welfare is weak, and the effect is similar across countries. The present study contributes to the general discussion currently underway about the definition of well-being and the connection between well-being and achievement. The results highlighted two complementary concerns: there is a clear need to promote socio-emotional education in schools, and it is important to develop a rigorous framework for well-being assessment. The implications of the results and proposals for future studies are discussed.
Validity evidence based on response processes was first introduced explicitly as a source of validity evidence in the latest edition of Standards for Educational and Psychological Testing. In this paper, we present the theory, the relationship with other sources of validity evidence, and the methods available for validation studies aimed at obtaining validity evidence about response processes. A comprehensive review of the literature along with theoretical and practical proposals. The articles provides arguments for determining when validity evidence based on response processes is critical for supporting the use of the test for a particular purpose, and examples of how to perform a validation study to obtain such validity evidence. There are methods for obtaining validity evidence based on response processes. Special attention should be paid to validation studies using the cognitive interview method given its features and possibilities. Future research problems pose how to combine data from different methods -qualitative and quantitative-, to develop complete validity arguments that support the use of the test for a particular purpose.
The latest trends in research extend the focus of school effectiveness beyond students’ acquisition of knowledge and skills, looking at aspects such as well-being in the academic context. Although the concept of well-being itself has been defined and measured in various ways, neither its dimensions nor the relationships between the components have been clearly described. The aim of the present study was to analyse how the elements of well-being interact and determine how they are influenced by school factors. To do that, we conducted a network analysis based on data from the Programme for International Student Assessment (PISA) 2018 international assessment. Our results demonstrated that cognitive, psychological, and social well-being variables form a solid welfare construct in the educational context, where students’ resilience and fear of failure, along with their sense of belonging, play central roles. Although the influence of school factors on student well-being is generally low, teaching enthusiasm and support promote positive school climates which are, in turn, crucial in reducing bullying.
Background:: Validity is a core topic in educational and psychological assessment. Although there are many available resources describing the concept of validity, sources of validity evidence, and suggestions about how to obtain validity evidence; there is little guidance providing specific instructions for planning and carrying out validation studies. Method:: In this paper we describe (a) the fundamental principles underlying test validity, (b) the process of validation, and (c) practical guidance for practitioners to plan and carry out sufficient validity research to support the use of a test for its intended purposes. Results:: We first define validity, describe sources of validity evidence, and provide examples where each of these sources are addressed. Then, we describe a validation agenda describing steps and tasks for planning and developing validation studies. Conclusions:: Finally, we discuss the importance of addressing validation studies from a comprehensive approach.
Background: Primary health care (PHC) professionals may play a crucial role in improving early diagnosis of depressive disorders. However, only 50% of cases are detected in PHC. The most widely used screening instrument for major depression is the Patient Health Questionnaire (PHQ), including the two-, eight- and nine-item versions. Surprisingly, there is neither enough evidence about the validity of PHQ in PHC patients in Spain nor indications about how to interpret the total scores. This study aimed to gather validity evidence to support the use of the three PHQ versions to screen for major depression in PHC in Spain. Additionally, the present study provided information for helping professionals to choose the best PHQ version according to the context.Methods: The sample was composed of 2579 participants from 22 Spanish PHC centers participating in the EIRA-3 study. The reliability and validity of the three PHQ versions for Spanish PHC patients were assessed based on responses to the questionnaire.Results: The PHQ-8 and PHQ-9 showed high internal consistency. The results obtained confirm the theoretically expected relationship between PHQ results and anxiety, social support and health-related QoL. A single-factor solution was confirmed. Regarding to the level of agreement with the CIDI interview (used as the criterion), our results indicate that the PHQ has a good discrimination power. The optimal cut-off values were: ⩾2 for PHQ-2, ⩾7 for PHQ-8 and ⩾8 for PHQ-9.Conclusions: PHQ is a good and valuable tool for detecting major depression in PHC patients in Spain.
1 November 2018 in Neuroscience & Biobehavioral Reviews