Systematic reviews and meta-analyses are crucial for advancing research, yet they are time-consuming and resource-demanding. Although machine learning and natural language processing algorithms may reduce this time and these resources, their performance has not been tested in education and educational psychology, and there is a lack of clear information on when researchers should stop the reviewing process. In this study, we conducted a retrospective screening simulation using 27 systematic reviews in education and educational psychology. We evaluated the sensitivity, specificity, and estimated time savings of several learning algorithms and heuristic stopping criteria. The results showed, on average, a 58% (SD = 19%) reduction in the screening workload of irrelevant records when using learning algorithms for abstract screening and an estimated time savings of 1.66 days (SD = 1.80). The learning algorithm random forests with sentence bidirectional encoder representations from transformers outperformed other algorithms. This finding emphasizes the importance of incorporating semantic and contextual information during feature extraction and modeling in the screening process. Furthermore, we found that 95% of all relevant abstracts within a given dataset can be retrieved using heuristic stopping rules. Specifically, an approach that stops the screening process after classifying 20% of records and consecutively classifying 5% of irrelevant papers yielded the most significant gains in terms of specificity (M = 42%, SD = 28%). However, the performance of the heuristic stopping criteria depended on the learning algorithm used and the length and proportion of relevant papers in an abstract collection. Our study provides empirical evidence on the performance of machine learning screening algorithms for abstract screening in systematic reviews in education and educational psychology.

This journal recently published a systematic review of simulation studies on the performance of Bayesian approaches for estimating latent variable models in small samples. The authors of this review highlighted that Bayesian approaches can perform poorly (i.e., by exhibiting bias) when the prior distributions are not thoughtfully constructed on the basis of previous knowledge. In this comment, we question whether the bias is the most important criterion when the sample size is small. We argue that the variability is more important and should therefore not be ignored. Moreover, because one of the most important selling points of Bayesian approaches was not addressed in the article, we argue that although somewhat biased, Bayesian approaches allow for more accurate estimates (i.e., a smaller mean squared error) than Maximum Likelihood (ML) in small samples, and we show one such approach that is more accurate than ML.

Autoregressive modeling has traditionally been concerned with time-series data from one unit (N = 1). For short time series (T < 50), estimation performance problems are well studied and documented. Fortunately, in psychological and social science research, besides T, another source of information is often available for model estimation, that is, the persons (N > 1). In this work, we illustrate the N/T compensation effect: With an increasing number of persons N at constant T, the model estimation performance increases, and vice versa, with an increasing number of time points T at constant N, the performance increases as well. Based on these observations, we develop sample size recommendations in the form of easily accessible N/T heatmaps for two popular autoregressive continuous-time models.

The COVID-19 pandemic and associated lockdowns have posed unique and severe challenges to our global society. To gain an integrative understanding of pervasive social and mental health impacts in 3522 Berlin residents aged 18 to 65, we systematically investigated the structural and temporal relationship between a variety of psychological indicators of vulnerability, resilience and social cohesion before, during and after the first lockdown in Germany using a retrospective longitudinal study design. Factor analyses revealed that (a) vulnerability and resilience indicators converged on one general bipolar factor, (b) residual variance of resilience indicators formed a distinct factor of adaptive coping capacities and (c) social cohesion could be reliably measured with a hierarchical model including four first-order dimensions of trust, a sense of belonging, social interactions and social engagement, and one second-order social cohesion factor. In the second step, latent change score models revealed that overall psychological vulnerability increased during the first lockdown and decreased again during re-opening, although not to baseline levels. Levels of social cohesion, in contrast, first decreased and then increased again during re-opening. Furthermore, participants who increased in vulnerability simultaneously decreased in social cohesion and adaptive coping during lockdown. While higher pre-lockdown levels of social cohesion predicted a stronger lockdown effect on mental health, individuals with higher social cohesion during the lockdown and positive change in coping abilities and social cohesion during re-opening showed better mental health recovery, highlighting the important role of social capacities in both amplifying but also overcoming the multiple challenges of this collective crisis.

Background: The SARS-CoV-2 pandemic has led to a mental health crisis on a global scale. Epidemiological studies have reported a drastic increase in mental health problems, such as depression and anxiety, increased loneliness and feelings of disconnectedness from others, while resilience levels have been negatively affected, indicating an urgent need for intervention. The current study is embedded within the larger CovSocial project which sought to evaluate longitudinal changes in vulnerability, resilience and social cohesion during the pandemic. The current second phase will investigate the efficacy of brief online mental training interventions in reducing mental health problems, and enhancing psychological resilience and social capacities. It further provides a unique opportunity for the prediction of intervention effects by individual biopsychosocial characteristics and preceding longitudinal change patterns during the pandemic in 2020/21.Methods: We will examine the differential effects of a socio-emotional (including ‘Affect Dyad’) and a mindfulness-based (including ‘Breathing Meditation’) intervention, delivered through a web- and cellphone application. Participants will undergo 10 weeks of intervention, and will be compared to a retest control group. The effectiveness of the interventions will be evaluated in a community sample (N= 300), which is recruited from the original longitudinal CovSocial sample. The pre- to post-intervention changes, potential underlying mechanisms, and prediction thereof, will be assessed on a wide range of outcomes: levels of stress, loneliness, depression and anxiety, resilience, prosocial behavior, empathy, compassion, and the impact on neuroendocrine, immunological and epigenetic markers. The multi-method nature of the study will incorporate self-report questionnaires, behavioral tasks, ecological momentary assessment (EMA) approaches, and biological, hormonal and epigenetic markers assessed in saliva.Discussion: Results will reveal the differential effectiveness of two brief online interventions in improving mental health outcomes, as well as enhancing social capacities and resilience. The present study will serve as a first step for future application of scalable, low-cost interventions at a broader level to reduce stress and loneliness, improve mental health and build resilience and social capacities in the face of global stressors.Trial registration: This trial has been registered on May 17, 2020 with the ClinicalTrials.govNCT04889508registration number (clinicaltrials.gov/ct2/show/NCT04889508).

Several AI‐aided screening tools have emerged to tackle the ever‐expanding body of literature. These tools employ active learning, where algorithms sort abstracts based on human feedback. However, researchers using these tools face a crucial dilemma: When should they stop screening without knowing the proportion of relevant studies? Although numerous stopping rules have been proposed to guide users in this decision, they have yet to undergo comprehensive evaluation. In this study, we evaluated the performance of three stopping rules: the knee method, a data‐driven heuristic, and a prevalence estimation technique. We measured performance via sensitivity, specificity, and screening cost and explored the influence of the prevalence of relevant studies and the choice of the learning algorithm. We curated a dataset of abstract collections from meta‐analyses across five psychological research domains. Our findings revealed performance differences between stopping rules regarding all performance measures and variations in the performance of stopping rules across different prevalence ratios. Moreover, despite the relatively minor impact of the learning algorithm, we found that specific combinations of stopping rules and learning algorithms were most effective for certain prevalence ratios of relevant abstracts. Based on these results, we derived practical recommendations for users of AI‐aided screening tools. Furthermore, we discuss possible implications and offer suggestions for future research.