Research linking the “quiet eye” (QE) period to subsequent performance has not been systematically synthesized. In this paper we review the literature on the link between the two through nonintervention (Synthesis 1) and intervention (Synthesis 2) studies. In the first synthesis, 27 studies with 38 effect sizes resulted in a large mean effect (d = 1.04) reflecting differences between experts’ and novices’ QE periods, and a moderate effect size (d = 0.58) comparing QE periods for successful and unsuccessful performances within individuals. Studies reporting QE duration as a percentage of the total time revealed a larger mean effect size than studies reporting an absolute duration (in milliseconds). The second synthesis of 9 articles revealed very large effect sizes for both the quiet-eye period (d = 1.53) and performance (d = 0.84). QE also showed some ability to predict performance effects across studies.

Indirect observation is a recent concept in systematic observation. It largely involves analyzing textual material generated either indirectly from transcriptions of audio recordings of verbal behavior in natural settings (e.g., conversation, group discussions) or directly from narratives (e.g., letters of complaint, tweets, forum posts). It may also feature seemingly unobtrusive objects that can provide relevant insights into daily routines. All these materials constitute an extremely rich source of information for studying everyday life, and they are continuously growing with the burgeoning of new technologies for data recording, dissemination, and storage. Narratives are an excellent vehicle for studying everyday life, and quantitization is proposed as a means of integrating qualitative and quantitative elements. However, this analysis requires a structured system that enables researchers to analyze varying forms and sources of information objectively. In this paper, we present a methodological framework detailing the steps and decisions required to quantitatively analyze a set of data that was originally qualitative. We provide guidelines on study dimensions, text segmentation criteria, ad hoc observation instruments, data quality controls, and coding and preparation of text for quantitative analysis. The quality control stage is essential to ensure that the code matrices generated from the qualitative data are reliable. We provide examples of how an indirect observation study can produce data for quantitative analysis and also describe the different software tools available for the various stages of the process. The proposed method is framed within a specific mixed methods approach that involves collecting qualitative data and subsequently transforming these into matrices of codes (not frequencies) for quantitative analysis to detect underlying structures and behavioral patterns. The data collection and quality control procedures fully meet the requirement of flexibility and provide new perspectives on data integration in the study of biopsychosocial aspects in everyday contexts.

Observational methodology is one of the most suitable research designs for evaluating fidelity of implementation, especially in complex interventions. However, the conduct and reporting of observational studies is hampered by the absence of specific guidelines, such as those that exist for other evaluation designs. This lack of specific guidance poses a threat to the quality and transparency of these studies and also constitutes a considerable publication hurdle. The aim of this study thus was to draw up a set of proposed guidelines for reporting evaluations based on observational methodology. The guidelines were developed by triangulating three sources of information: observational studies performed in different fields by experts in observational methodology, reporting guidelines for general studies and studies with similar designs to observational studies, and proposals from experts in observational methodology at scientific meetings. We produced a list of guidelines grouped into three domains: intervention and expected outcomes, methods, and results. The result is a useful, carefully crafted set of simple guidelines for conducting and reporting observational studies in the field of program evaluation.

Studies of the dimensionality of the Life Orientation Test-Revised (LOT-R), considered as the gold standard in the measurement of dispositional optimism, yield controversial results due to the various factorial solutions found. Consequently, the factorial structure of the test has not yet been fully established. The aim of this study is to determine the factorial structure of the LOT-R by comparing seven previous models and their empirical evidence. The test was administered to 906 Spanish participants, ages 18 to 61 (mean age: 23; 56% males). Confirmatory factor analyses were conducted using polychoric correlations. Considering the theoretical background and the best model fit indices (RMSEA=.038; CFI=.98), we conclude that the test presents a factorial structure of a second-order factor (life orientation) composed of two factors (optimism and pessimism). Thus, we recommend using a single global score that could be referred to as life orientation but which ultimately represents the level of dispositional optimism.

Two-phase single-case designs, including baseline evaluation followed by an intervention, represent the most clinically straightforward option for combining professional practice and research. However, unless they are part of a multiple-baseline schedule, such designs do not allow demonstrating a causal relation between the intervention and the behavior. Although the statistical options reviewed here cannot help overcoming this methodological limitation, we aim to make practitioners and applied researchers aware of the available appropriate options for extracting maximum information from the data. In the current paper, we suggest that the evaluation of behavioral change should include visual and quantitative analyses, complementing the substantive criteria regarding the practical importance of the behavioral change. Specifically, we emphasize the need to use structured criteria for visual analysis, such as the ones summarized in the What Works Clearinghouse Standards, especially if such criteria are complemented by visual aids, as illustrated here. For quantitative analysis, we focus on the Nonoverlap of all pairs and the Slope and level change procedure, as they offer straightforward information and have shown reasonable performance. An illustration is provided of the use of these three pieces of information: visual, quantitative, and substantive. To make the use of visual and quantitative analysis feasible, open source software is referred to and demonstrated. In order to provide practitioners and applied researchers with a more complete guide, several analytical alternatives are commented on pointing out the situations (aims, data patterns) for which these are potentially useful.

We describe the process of testing a measurement model of a satisfaction rating scale for use with training programs. The scale was developed by the Evaluation Unit of the University of Seville's Training Center. Following an initial content validity study in which we reviewed how training evaluation is modeled, a 12-item rating scale was developed and administered to a sample of 2746 subjects. The measurement model was examined through an exploratory factor analysis using polychoric correlations, the results of which were consistent with the previous dimensions obtained in the content validity study. In order to refine the model under study, a confirmatory factor analysis (also based on polychoric correlations) was then performed. The results present empirical evidence of the measurement model's validity. The scale offers simple, direct and reliable training satisfaction scores. Future development of the scale is discussed.