Background: Despite the growing interest in the experience sampling method (ESM) as a data collection tool for mental health research, the absence of methodological guidelines related to its use has resulted in a large heterogeneity of designs. Concomitantly, the potential effects of the design on the response behavior of the participants remain largely unknown. Objective: The objective of this meta-analysis was to investigate the associations between various sample and design characteristics and the compliance and retention rates of studies using ESM in mental health research. Methods: ESM studies investigating major depressive disorder, bipolar disorder, and psychotic disorder were considered for inclusion. Besides the compliance and retention rates, a number of sample and design characteristics of the selected studies were collected to assess their potential relationships with the compliance and retention rates. Multilevel random/mixed effects models were used for the analyses. Results: Compliance and retention rates were lower for studies with a higher proportion of male participants (P<.001) and individuals with a psychotic disorder (P<.001). Compliance was positively associated with the use of a fixed sampling scheme (P=.02), higher incentives (P=.03), higher time intervals between successive evaluations (P=.02), and fewer evaluations per day (P=.008), while no significant associations were observed with regard to the mean age of the sample, the study duration, or other design characteristics. Conclusions: The findings demonstrate that ESM studies can be carried out in mental health research, but the quality of the data collection depends upon a number of factors related to the design of ESM studies and the samples under study that need to be considered when designing such protocols.
Since the introduction of the experience sampling method (ESM), there have been concerns that the repeated assessments typically related to this method may alter the behavior, thoughts, or feelings of participants. Previous studies have offered mixed results with some studies reporting reactive changes, while others failed to find such effects. Our aim was to investigate under which circumstances ESM induces reactive effects. Students (N = 151) were randomly assigned to receive a questionnaire containing 30 or 60 items three, six, or nine times per day for 14 days. A random sample of 50 participants took part in qualitative interviews after the end of the data collection. We investigated changes over time in the data, while taking into account the sampling protocol and characteristics of participants, and analyzed qualitative reports of measurement reactivity. Decreases in completion time, within-person variance of ratings and subjective reports of habituation point toward the existence of a habituation period. While participants reported increases in emotional awareness in interviews, ESM measures indicated a decrease in emotional awareness over time. Changes in behavior were rare in quantitative and qualitative reports. Positive affect was decreasing over time in the ESM data, and various changes in affect, emotion regulation, and thoughts were reported in interviews. Individual characteristics and sampling protocol had inconsistent effects on changes over time. The results suggest that ESM induces changes in within-person variability, completion times, affect, or emotional awareness over time. Further research is needed to explore whether observed changes affect the validity of ESM data. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
1 September 2013 in Personality and Individual Differences