Jesse J. Chandler

Jesse J. Chandler

Auto-generated profile, based on publicly available information

Jesse J. Chandler

University of Michigan, Ann Arbor, USA

Research Focus & Metrics

Academic Career Length

19Years (2008-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 March 2016 in Annual Review of Clinical Psychology
Verified
Journal Article
Closed access
JCJesse Chandler
Jesse Chandler
DSDanielle Shapiro
Danielle Shapiro
Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk—many of which are common to other nonprobability samples but unfamiliar to clinical science researchers—and suggests concrete steps to avoid these issues or minimize their impact.

9 July 2013 in Behavior Research Methods
Verified
Journal Article
Closed access
PDF file
JCJesse Chandler
Jesse Chandler
PMPam Mueller
Pam Mueller
GPGabriele Paolacci
Gabriele Paolacci

24 December 2018 in Advances in Methods and Practices in Psychological Science
Verified
Journal Article
Open access
RKRichard A. Klein
Richard A. Klein
MVMichelangelo Vianello
Michelangelo Vianello
FHFred Hasselman
Fred Hasselman
BAByron G. Adams
Byron G. Adams
RAReginald B. Adams
Reginald B. Adams
BNBrian A. Nosek
Brian A. Nosek
We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.

27 April 2017 in Social Psychological and Personality Science
Verified
Journal Article
Closed access
JCJesse J. Chandler
Jesse J. Chandler
GPGabriele Paolacci
Gabriele Paolacci
The Internet has enabled recruitment of large samples with specific characteristics. However, when researchers rely on participant self-report to determine eligibility, data quality depends on participant honesty. Across four studies on Amazon Mechanical Turk, we show that a substantial number of participants misrepresent theoretically relevant characteristics (e.g., demographics, product ownership) to meet eligibility criteria explicit in the studies, inferred by a previous exclusion from the study or inferred in previous experiences with similar studies. When recruiting rare populations, a large proportion of responses can be impostors. We provide recommendations about how to ensure that ineligible participants are excluded that are applicable to a wide variety of data collection efforts, which rely on self-report.

1 October 2017 in Trends in Cognitive Sciences
Verified
Journal Article
Open access
NSNeil Stewart
Neil Stewart
JCJesse Chandler
Jesse Chandler
GPGabriele Paolacci
Gabriele Paolacci
Crowdsourcing data collection from research participants recruited from online labor markets is now common in cognitive science. We review who is in the crowd and who can be reached by the average laboratory. We discuss reproducibility and review some recent methodological innovations for online experiments. We consider the design of research studies and arising ethical issues. We review how to code experiments for the web, what is known about video and audio presentation, and the measurement of reaction times. We close with comments about the high levels of experience of many participants and an emerging tragedy of the commons.

10 June 2015 in Psychological Science
Verified
Journal Article
Closed access
JCJesse Chandler
Jesse Chandler
GPGabriele Paolacci
Gabriele Paolacci
EPEyal Peer
Eyal Peer
PMPam Mueller
Pam Mueller
KRKate A. Ratliff
Kate A. Ratliff
Although researchers often assume their participants are naive to experimental materials, this is not always the case. We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in the first. The reduction was most pronounced when participants were assigned to a different condition in the second wave. We discuss the methodological implications of these findings.