Benjamin George Farrar

Benjamin George Farrar

Auto-generated profile, based on publicly available information

Benjamin George Farrar

Department of Psychology, University of Cambridge

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 February 2020 in Animal Behavior and Cognition
Verified
Journal Article
Open access
BFBenjamin Farrar
Benjamin Farrar
MBMarkus Boeckle
Markus Boeckle
NCNicola Clayton
Nicola Clayton
Direct replication studies follow an original experiment’s methods as closely as possible. They provide information about the reliability and validity of an original study’s findings. The present paper asks what comparative cognition should expect if its studies were directly replicated, and how researchers can use this information to improve the reliability of future research. Because published effect sizes are likely overestimated, comparative cognition researchers should not expect findings with p-values just below the significance level to replicate consistently. Nevertheless, there are several statistical and design features that can help researchers identify reliable research. However, researchers should not simply aim for maximum replicability when planning studies; comparative cognition faces strong replicability-validity and replicability-resource trade-offs. Next, the paper argues that it may not even be possible to perform truly direct replication studies in comparative cognition because of: 1) a lack of access to the species of interest; 2) real differences in animal behavior across sites; and 3) sample size constraints producing very uncertain statistical estimates, meaning that it will often not be possible to detect statistical differences between original and replication studies. These three reasons suggest that many claims in the comparative cognition literature are practically unfalsifiable, and this presents a challenge for cumulative science in comparative cognition. To address this challenge, comparative cognition can begin to formally assess the replicability of its findings, improve its statistical thinking and explore new infrastructures that can help to form a field that can create and combine the data necessary to understand how cognition evolves.

1 May 2021 in Animal Behavior and Cognition
Verified
Journal Article
Open access
BFBenjamin G. Farrar
Benjamin G. Farrar
KVKonstantinos Voudouris
Konstantinos Voudouris
NCNicola S. Clayton
Nicola S. Clayton
Animal cognition research often involves small and idiosyncratic samples. This can constrain the generalizability and replicability of a study’s results and prevent meaningful comparisons between samples. However, there is little consensus about what makes a strong replication or comparison in animal research. We apply a resampling definition of replication to answer these questions in Part 1 of this article, and, in Part 2, we focus on the problem of representativeness in animal research. Through a case study and a simulation study, we highlight how and when representativeness may be an issue in animal behavior and cognition research and show how the representativeness problems can be viewed through the lenses of, i) replicability, ii) generalizability and external validity, iii) pseudoreplication and, iv) theory testing. Next, we discuss when and how researchers can improve their ability to learn from small sample research through, i) increasing heterogeneity in experimental design, ii) increasing homogeneity in experimental design, and, iii) statistically modeling variation. Finally, we describe how the strongest solutions will vary depending on the goals and resources of individual research programs and discuss some barriers towards implementing them.

1 August 2020 in Animal Behavior and Cognition
Verified
Journal Article
Closed access
BFBenjamin G. Farrar
Benjamin G. Farrar
DADrew M. Altschul
Drew M. Altschul
JFJulia Fischer
Julia Fischer
JMJolene van der Mescht
Jolene van der Mescht
SPSarah Placi
Sarah Placi
LOLjerka Ostojic
Ljerka Ostojic
Scientific disciplines face concerns about replicability and statistical inference, and these concerns are also relevant in animal cognition research. This paper presents a first attempt to assess how researchers make and publish claims about animal physical cognition, and the statistical inferences they use to support them. We surveyed 116 published experiments from 63 papers on physical cognition, covering 43 different species. The most common tasks in our sample were trap-tube tasks (14 papers), other tool use tasks (13 papers), means-end understanding and string-pulling tasks (11 papers), object choice and object permanence tasks (9 papers) and access tasks (5 papers). This sample is not representative of the full scope of physical cognition research; however, it does provide data on the types of statistical design and publication decisions researchers have adopted. Across the 116 experiments, the median sample size was 7. Depending on the definitions we used, we estimated that between 44% and 59% of our sample of papers made positive claims about animals’ physical cognitive abilities, between 24% and 46% made inconclusive claims, and between 10% and 17% made negative claims. Several failures of animals to pass physical cognition tasks were reported. Although our measures had low inter-observer reliability, these findings show that negative results can and have been published in the field. However, publication bias is still present, and consistent with this, we observed a drop in the frequency of p-values above .05. This suggests that some non-significant results have not been published. More promisingly, we found that researchers are likely making many correct statistical inferences at the individual-level. The strength of evidence of statistical effects at the group-level was weaker, and its p-value distribution was consistent with some effect sizes being overestimated. Studies such as ours can form part of a wider investigation into statistical reliability in comparative cognition. However, future work should focus on developing the validity and reliability of the measurements they use, and we offer some starting points.

2 October 2019 in PsyArXiv
Verified
Preprint
Open access
BFBenjamin George Farrar
Benjamin George Farrar
LOLjerka Ostojic
Ljerka Ostojic
A prominent vein of comparative cognition research asks which cognitive abilities may be ascribed to different species. Here, we argue that the current structure of comparative cognition makes it near impossible to evaluate the accuracy of many of the claims produced by the field’s empirical research. We base this argument on six observations: i) the field is biased towards confirming more exceptional abilities in animals, ii) a likely high rate of false positive discovery, iii) the persistence of the bias towards more exceptional abilities, even in the presence of strong methodological criticism, iv) the absence of a formal method to assess evidence of absence of a cognitive ability, v) the ambiguity in definitions used to make claims, and vi) the small size of the research field. We then highlight how the widespread methodological and conceptual criticism in the comparative cognition literature does not facilitate scientific progress, rather it can impede it by creating an illusion of scientific rigour. Finally, we call for a reform of our research field that draws on recent developments in related fields and the Open Science movement.

31 August 2021 in PLOS ONE
Verified
Journal Article
Open access
PDF file
BFBenjamin G. Farrar
Benjamin G. Farrar
LOLjerka Ostojić
Ljerka Ostojić
NCNicola S. Clayton
Nicola S. Clayton
Animal cognition research aims to understand animal minds by using a diverse range of methods across an equally diverse range of species. Throughout its history, the field has sought to mitigate various biases that occur when studying animal minds, from experimenter effects to anthropomorphism. Recently, there has also been a focus on how common scientific practices might affect the reliability and validity of published research. Usually, these issues are discussed in the literature by a small group of scholars with a specific interest in the topics. This study aimed to survey a wider range of animal cognition researchers to ask about their attitudes towards classic and contemporary issues facing the field. Two-hundred and ten active animal cognition researchers completed our survey, and provided answers on questions relating to bias, replicability, statistics, publication, and belief in animal cognition. Collectively, researchers were wary of bias in the research field, but less so in their own work. Over 70% of researchers endorsed Morgan’s canon as a useful principle but many caveated this in their free-text responses. Researchers self-reported that most of their studies had been published, however they often reported that studies went unpublished because they had negative or inconclusive results, or results that questioned “preferred” theories. Researchers rarely reported having performed questionable research practices themselves—however they thought that other researchers sometimes (52.7% of responses) or often (27.9% of responses) perform them. Researchers near unanimously agreed that replication studies are important but too infrequently performed in animal cognition research, 73.0% of respondents suggested areas of animal cognition research could experience a ‘replication crisis’ if replication studies were performed. Consistently, participants’ free-text responses provided a nuanced picture of the challenges animal cognition research faces, which are available as part of an open dataset. However, many researchers appeared concerned with how to interpret negative results, publication bias, theoretical bias and reliability in areas of animal cognition research. Collectively, these data provide a candid overview of barriers to progress in animal cognition and can inform debates on how individual researchers, as well as organizations and journals, can facilitate robust scientific research in animal cognition.

21 January 2020 in PsyArXiv
Verified
Preprint
Open access
BFBenjamin George Farrar
Benjamin George Farrar
Fayet, Hansen and Biro (1) provide two observations of Atlantic puffins, Fratercula arctica performing self-directed actions while holding a stick in their beaks. The authors interpret this as evidence of tool use as they suggest that the stick was manipulated towards the birds’ plumage with the specific goal of scratching. While these are interesting observations, the authors do not address a simple and likely explanation of the self-directed actions that does not constitute tool use; that the observations occurred by chance. As such, their conclusions about tool use and cognition in seabirds are premature.