Justin M. Calabrese

Dr. Justin M. Calabrese


Research Focus & Metrics

Academic Career Length

17Years (2010-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 May 2016 in Methods in Ecology and Evolution
Verified
Journal Article
Closed access
PDF file
JCJustin M. Calabrese
Justin M. Calabrese
CFChris H. Fleming
Chris H. Fleming
EGEliezer Gurarie
Eliezer Gurarie

11 June 2018 in Ecological Monographs
Verified
Journal Article
Open access
PDF file
CDCarsten F. Dormann
Carsten F. Dormann
JCJustin M. Calabrese
Justin M. Calabrese
GGGurutzeta Guillera‐Arroita
Gurutzeta Guillera‐Arroita
EMEleni Matechou
Eleni Matechou
VBVolker Bahn
Volker Bahn
FHFlorian Hartig
Florian Hartig
In ecology, the true causal structure for a given problem is often not known, and several plausible models and thus model predictions exist. It has been claimed that using weighted averages of these models can reduce prediction error, as well as better reflect model selection uncertainty. These claims, however, are often demonstrated by isolated examples. Analysts must better understand under which conditions model averaging can improve predictions and their uncertainty estimates. Moreover, a large range of different model averaging methods exists, raising the question of how they differ in their behaviour and performance. Here, we review the mathematical foundations of model averaging along with the diversity of approaches available. We explain that the error in model‐averaged predictions depends on each model's predictive bias and variance, as well as the covariance in predictions between models, and uncertainty about model weights. We show that model averaging is particularly useful if the predictive error of contributing model predictions is dominated by variance, and if the covariance between models is low. For noisy data, which predominate in ecology, these conditions will often be met. Many different methods to derive averaging weights exist, from Bayesian over information‐theoretical to cross‐validation optimized and resampling approaches. A general recommendation is difficult, because the performance of methods is often context dependent. Importantly, estimating weights creates some additional uncertainty. As a result, estimated model weights may not always outperform arbitrary fixed weights, such as equal weights for all models. When averaging a set of models with many inadequate models, however, estimating model weights will typically be superior to equal weights. We also investigate the quality of the confidence intervals calculated for model‐averaged predictions, showing that they differ greatly in behaviour and seldom manage to achieve nominal coverage. Our overall recommendations stress the importance of non‐parametric methods such as cross‐validation for a reliable uncertainty quantification of model‐averaged predictions.

11 August 2013 in Global Ecology and Biogeography
Verified
Journal Article
Closed access
PDF file
JCJustin M. Calabrese
Justin M. Calabrese
GCGrégoire Certain
Grégoire Certain
CKCasper Kraan
Casper Kraan
CDCarsten F. Dormann
Carsten F. Dormann

29 November 2018 in Ecological Monographs
Verified
Journal Article
Open access
PDF file
MNMichael Noonan
Michael Noonan
MTMarlee A. Tucker
Marlee A. Tucker
CFChristen H. Fleming
Christen H. Fleming
TAThomas S. Akre
Thomas S. Akre
SASusan C. Alberts
Susan C. Alberts
JCJustin M. Calabrese
Justin M. Calabrese
Home range estimation is routine practice in ecological research. While advances in animal tracking technology have increased our capacity to collect data to support home range analysis, these same advances have also resulted in increasingly autocorrelated data. Consequently, the question of which home range estimator to use on modern, highly autocorrelated tracking data remains open. This question is particularly relevant given that most estimators assume independently sampled data. Here, we provide a comprehensive evaluation of the effects of autocorrelation on home range estimation. We base our study on an extensive data set of GPS locations from 369 individuals representing 27 species distributed across five continents. We first assemble a broad array of home range estimators, including Kernel Density Estimation (KDE) with four bandwidth optimizers (Gaussian reference function, autocorrelated‐Gaussian reference function [AKDE], Silverman's rule of thumb, and least squares cross‐validation), Minimum Convex Polygon, and Local Convex Hull methods. Notably, all of these estimators except AKDE assume independent and identically distributed (IID) data. We then employ half‐sample cross‐validation to objectively quantify estimator performance, and the recently introduced effective sample size for home range area estimation ( <!--${MathJax: TeX-AMS-MML_HTMLorMML}--> N ^ area ) to quantify the information content of each data set. We found that AKDE 95% area estimates were larger than conventional IID‐based estimates by a mean factor of 2. The median number of cross‐validated locations included in the hold‐out sets by AKDE 95% (or 50%) estimates was 95.3% (or 50.1%), confirming the larger AKDE ranges were appropriately selective at the specified quantile. Conversely, conventional estimates exhibited negative bias that increased with decreasing <!--${MathJax: TeX-AMS-MML_HTMLorMML}--> N ^ area . To contextualize our empirical results, we performed a detailed simulation study to tease apart how sampling frequency, sampling duration, and the focal animal's movement conspire to affect range estimates. Paralleling our empirical results, the simulation study demonstrated that AKDE was generally more accurate than conventional methods, particularly for small <!--${MathJax: TeX-AMS-MML_HTMLorMML}--> N ^ area . While 72% of the 369 empirical data sets had >1,000 total observations, only 4% had an <!--${MathJax: TeX-AMS-MML_HTMLorMML}--> N ^ area >1,000, where 30% had an <!--${MathJax: TeX-AMS-MML_HTMLorMML}--> N ^ area <30. In this frequently encountered scenario of small <!--${MathJax: TeX-AMS-MML_HTMLorMML}--> N ^ area , AKDE was the only estimator capable of producing an accurate home range estimate on autocorrelated data.

10 December 2021 in Methods in Ecology and Evolution
Verified
Journal Article
Open access
PDF file
ISInês Silva
Inês Silva
CFChristen H. Fleming
Christen H. Fleming
MNMichael Noonan
Michael Noonan
JAJesse Alston
Jesse Alston
CFCody Folta
Cody Folta
JCJustin M. Calabrese
Justin M. Calabrese
Modern tracking devices allow for the collection of high‐volume animal tracking data at improved sampling rates over very‐high‐frequency radiotelemetry. Home range estimation is a key output from these tracking datasets, but the inherent properties of animal movement can lead traditional statistical methods to under‐ or overestimate home range areas. The autocorrelated kernel density estimation (AKDE) family of estimators was designed to be statistically efficient while explicitly dealing with the complexities of modern movement data: autocorrelation, small sample sizes and missing or irregularly sampled data. Although each of these estimators has been described in separate technical papers, here we review how these estimators work and provide a user‐friendly guide on how they may be combined to reduce multiple biases simultaneously. We describe the magnitude of the improvements offered by these estimators and their impact on home range area estimates, using both empirical case studies and simulations, contrasting their computational costs. Finally, we provide guidelines for researchers to choose among alternative estimators and anRscript to facilitate the application and interpretation of AKDE home range estimates.

14 May 2018 in Methods in Ecology and Evolution
Verified
Journal Article
Closed access
PDF file
KWKevin Winner
Kevin Winner
MNMichael Noonan
Michael Noonan
CFChristen H. Fleming
Christen H. Fleming
KOKirk A. Olson
Kirk A. Olson
TMThomas Mueller
Thomas Mueller
JCJustin M. Calabrese
Justin M. Calabrese