Charles T. Gray

Charles T. Gray

Auto-generated profile, based on publicly available information

Charles T. Gray

La Trobe University

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications

22 February 2021 in MetaArXiv
Verified
Preprint
Open access
HFHannah Fraser
Hannah Fraser
MBMartin Bush
Martin Bush
BWBonnie Wintle
Bonnie Wintle
FMFallon Mody
Fallon Mody
ESEden T. Smith
Eden T. Smith
FFFiona Fidler
Fiona Fidler
Replication is a hallmark of scientific research. As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce a new technique to evaluating replicability, the repliCATS (Collaborative Assessments for Trustworthy Science) process, a structured expert elicitation approach based on the IDEA protocol. The repliCATS process is delivered through an underpinning online platform and applied to the evaluation of research claims in social and behavioural sciences. This process can be deployed for both rapid assessment of small numbers of claims, and assessment of high volumes of claims over an extended period. Pilot data suggests that the accuracy of the repliCATS process meets or exceeds that of other techniques used to predict replicability. An important advantage of the repliCATS process is that it collects qualitative data that has the potential to assist with problems like understanding the limits of generalizability of scientific claims. The repliCATS process has potential applications in alternative peer review and in the allocation of effort for replication studies.

1 April 2026 in MetaArXiv
Verified
Preprint
Open access
FMFallon Mody
Fallon Mody
DWDavid Peter Wilkinson
David Peter Wilkinson
HFHannah Fraser
Hannah Fraser
FFFiona Fidler
Fiona Fidler
MBMichael Metcalf Bishop
Michael Metcalf Bishop
TPThomas Pfeiffer
Thomas Pfeiffer
This paper reports two approaches to forecasting replicability for a corpus of 3000 published social science papers, using large-scale human assessments. Replication Markets used incentivized surveys and prediction markets where participants traded assets linked to replication outcomes and market prices were interpreted as replication predictions. The repliCATS project used a structured group deliberation protocol, including interactive discussion, and mathematically aggregated forecasts to generate replication predictions. Accuracy for both approaches was validated against a subset of (n=37) independent, high-power replication studies. The predictive accuracy (median [range]) achieved for AUC by Replication Markets was 0.76 [0.72-0.82], and 0.76 [0.68-0.81] by repliCATS. Replication Markets achieved a classification accuracy of 73% [68-76%] and repliCATS achieved 68% [59-78%]. These results place the performance of both teams within the accuracy range achieved in prior replication forecasting studies. We conclude that informative forecasts can be elicited by both methods, but there are trade-offs between scale and accuracy.

23 February 2021 in MetaArXiv
Verified
Preprint
Open access
Anca HaneaAnca Hanea
Anca Hanea
DWDavid Peter Wilkinson
David Peter Wilkinson
MMMarissa McBride
Marissa McBride
ALAidan Lyon
Aidan Lyon
DRDon van Ravenzwaaij
Don van Ravenzwaaij
BWBonnie Wintle
Bonnie Wintle
Experts are often asked to represent their uncertainty as a subjective probability. Structured protocols offer a transparent and systematic way to elicit and combine probability judgements from multiple experts. As part of this process, experts are asked to individually estimate a probability (e.g., of a future event) which needs to be combined, or aggregated, into a final group prediction. The experts' judgements can be aggregated behaviourally (by striving for consensus), or mathematically (by using a mathematical rule to combine individual estimates). Mathematical rules (e.g., weighted linear combinations of judgments) provide an objective approach to aggregation. However, the choice of a rule is not straightforward, and the quality of the aggregated group judgement depends on it. The quality of an aggregation can be defined in terms of accuracy, calibration and informativeness. These measures can be used to compare different aggregation approaches and help decide on which aggregation produces the ``best” final prediction.In the ideal case, individual experts' performance (as probability assessors) would be scored on similar questions ahead of time, these scores translated into performance-based weights, and a performance-based weighted aggregation could then be used. When this is not possible though, several other aggregation methods, informed by measurable proxies for good performance, can be formulated and compared. Here, we develop a suite of aggregation methods, informed by previous experience and the available literature. Next, we investigate the relative performance of these aggregation methods using three datasets. Although the accuracy, calibration, and informativeness of the majority of methods are very similar, a couple of the aggregation methods consistently distinguish themselves as among the best or worst.

30 April 2021 in MetaArXiv
Verified
Preprint
Open access
EGElliot Gould
Elliot Gould
CGCharles T. Gray
Charles T. Gray
AWAaron Willcox
Aaron Willcox
RORose E O'DEa
Rose E O'DEa
RGRebecca Groenewegen
Rebecca Groenewegen
DWDavid Peter Wilkinson
David Peter Wilkinson
Structured protocols, such as the IDEA protocol, may be used to elicit expert judgments in the form of subjective probabilities from multiple experts. Judgments from individual experts about a particular phenomena must therefore be mathematically aggregated into a single prediction. The process of aggregation may be complicated when uncertainty bounds are elicited with a judgment, and also when there are several rounds of elicitation. This paper presents the new R package \pkg{aggreCAT}, which provides 22 unique aggregation methods for combining individual judgments into a single, probabilistic measure. The aggregation methods were developed as a part of the Defense Advanced Research Projects Agency (DARPA) ‘Systematizing Confidence in Open Research and Evidence’ (SCORE) programme, which aims to generate confidence scores or estimates of ‘claim credibility’ for 3000 research claims from the social and behavioural sciences. We provide several worked examples illustrating the underlying mechanics of the aggregation methods. We also describe a general workflow for using the software in practice to facilitate uptake of this software for appropriate use-cases.

4 July 2025 in SocArXiv
Verified
Preprint
Open access
GGGarima Gupta
Garima Gupta
AMAchili Mihu
Achili Mihu
CGCharles T. Gray
Charles T. Gray
MPMaya Pasgaard
Maya Pasgaard
Indigenous Peoples and Local Communities (IPLCs) manage large swathes of the world’s remaining biodiversity and are essential actors in achieving global sustainability goals. Yet, recognition and representation of IPLCs remain uneven across regions. Despite decades of stewardship, IPLCs in many parts of the world, including in South Asia, face systemic underrepresentation due to longstanding epistemic and structural injustices. In India, this invisibility is exacerbated by the lack of formal recognition of 'Indigenous Peoples'—instead, the state uses administrative category 'Scheduled Tribes', which does not align with international definitions. Drawing on four case studies from India, this paper illustrates how traditional governance systems offer viable, culturally embedded inclusive nature- and people-based approaches to sustainable transformative change. Our paper argues that beyond decolonizing conservation discourse, there is a need for reparative justice: to redress historical exclusion, restore land rights, and fund IPLC-led conservation directly. Recognition justice for South Asian IPLCs is a critical and often overlooked component of global environmental governance. India serves not as an exception but as a representative case of broader systemic gaps. Bridging these gaps is not only a matter of justice but an ecological necessity for achieving the Global Biodiversity Framework and the IPBES vision of transformative change.