Sound scale construction is pivotal to the measurement of psychological constructs. Common item sampling procedures emphasize aspects of reliability to the disadvantage of aspects of validity, which are less tangible. We use a health knowledge test as an example to demonstrate how item sampling strategies that focus on either factor saturation or construct coverage influence scale composition and demonstrate how to find a trade-off between these two opposing needs. More specifically, we compile three 75-item health knowledge scales using Ant Colony Optimization, a metaheuristic algorithm that is inspired by the foraging behavior of ants, to optimize factor saturation, construct coverage, or a compromise of both. We demonstrate that our approach is well suited to balance out construct coverage and factor saturation when constructing a health knowledge test. Finally, we discuss conceptual problems with the modeling of declarative knowledge and provide recommendations for the assessment of health knowledge.
Cheating is a serious threat in unproctored ability assessment, irrespective of countermeasures taken, anticipated consequences (high vs. low stakes), and test modality (paper-pencil vs. computer-based). In the present study, we examined the power of (a) self-report-based indicators (i.e., Honesty-Humility and Overclaiming scales), (b) test data (i.e., performance with extremely difficult items), and (c) para data (i.e., reaction times, switching between browser tabs) to predict participants’ cheating behavior. To this end, 315 participants worked on a knowledge test in an unproctored online assessment and subsequently in a proctored lab assessment. We used multiple regression analysis and an extended latent change score model to assess the potential of the different indicators to predict cheating. In summary, test data and para data performed best, while traditional self-report-based indicators were not predictive. We discuss the findings with respect to unproctored testing in general and provide practical advice on cheating detection in online ability assessments.
Suicide is a major global health concern and a prominent cause of death in adolescents. Previous research on suicide prediction has mainly focused on clinical or adult samples. To prevent suicides at an early stage, however, it is important to screen for risk factors in a community sample of adolescents. We compared the accuracy of logistic regressions, elastic net regressions, and gradient boosting machines in predicting suicide attempts by 17-year-olds in the Millennium Cohort Study ( N = 7,347), combining a large set of self- and other-reported variables from different categories. Both machine learning algorithms outperformed logistic regressions and achieved similar balanced accuracies (.76 when using data 3 years before the self-reported lifetime suicide attempts and .85 when using data from the same measurement wave). We identified essential variables that should be considered when screening for suicidal behavior. Finally, we discuss the usefulness of complex machine learning models in suicide prediction.
Creativity can be measured with a variety of methods including self-reports, others reports, and ability tests. While typical self-reports are best understood as weak proxies of creativity, biographical reports that assess previous creative activities seem more promising. Drawbacks of such measures – including skewed item distributions, a lack of measurement invariance across sex, and low convergent validity with established measures of creativity (such as divergent thinking tasks) – might be attributable to issues of item sampling. Therefore, we use a meta-heuristic algorithm (Ant Colony Optimization) to develop a short-scale of creative activities that a) maintains coverage of the original domains, b) provides good model fit, c) has good reliability, d) demonstrates balanced item difficulty, e) is measurement invariant across sex, and f) possesses convergent validity with divergent thinking. For an 8-item short version of the original Inventory of Creative Activities and Achievements, we identified an item set that satisfies the most pre-specified criteria using data from N = 296 adults. In a replication sample (N = 327) we replicate the findings and report good model fit, good reliability, adequate item difficulty, and measurement invariance across sex. We derive recommendations on the measurement of creative activities, and discuss implications for understanding creativity in general.
Chronic stress is an important predictor of mental and physical health, but little is known about its association with cognitive abilities and education during the lifespan. We hypothesized that chronic stress would be negatively correlated with cognitive abilities, particularly crystallized intelligence, and that this association would be stronger among individuals with lower educational attainment due to limited stress-coping resources. We used cross-sectional data from the German National Educational Panel Study (NEPS SC6), comprising 10,416 adults aged 29 to 71 years (50.80% female; 49.20% male). Fluid and crystallized intelligence were assessed with a reasoning test and a vocabulary test, respectively; chronic stress was assessed with a questionnaire on social stress and anxiety. The tests and the questionnaire were conceptualized for a heterogeneous and large-scale sample. Our results show small negative associations between chronic stress and both fluid and crystallized cognitive abilities, which persist after controlling for demographic variables. However, there were no significant differences between educational groups. Although the study does not address longitudinal patterns, it highlights the complex interaction between stress and cognition, and it underscores the need for further research to explore how educational resources may mitigate the impact of chronic stress on cognitive health.