Understanding Earth system dynamics in light of ongoing human intervention and dependency remains a major scientific challenge. The unprecedented availability of data streams describing different facets of the Earth now offers fundamentally new avenues to address this quest. However, several practical hurdles, especially the lack of data interoperability, limit the joint potential of these data streams. Today, many initiatives within and beyond the Earth system sciences are exploring new approaches to overcome these hurdles and meet the growing interdisciplinary need for data-intensive research; using data cubes is one promising avenue. Here, we introduce the concept of Earth system data cubes and how to operate on them in a formal way. The idea is that treating multiple data dimensions, such as spatial, temporal, variable, frequency, and other grids alike, allows effective application of user-defined functions to co-interpret Earth observations and/or model–data integration. An implementation of this concept combines analysis-ready data cubes with a suitable analytic interface. In three case studies, we demonstrate how the concept and its implementation facilitate the execution of complex workflows for research across multiple variables, and spatial and temporal scales: (1) summary statistics for ecosystem and climate dynamics; (2) intrinsic dimensionality analysis on multiple timescales; and (3) model–data integration. We discuss the emerging perspectives for investigating global interacting and coupled phenomena in observed or simulated data. In particular, we see many emerging perspectives of this approach for interpreting large-scale model ensembles. The latest developments in machine learning, causal inference, and model–data integration can be seamlessly implemented in the proposed framework, supporting rapid progress in data-intensive research across disciplinary boundaries.
7 July 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
We consider the problem of predicting a response $Y$ from a set of covariates $X$ when test- and training distributions differ. Since such differences may have causal explanations, we consider test distributions that emerge from interventions in a structural causal model, and focus on minimizing the worst-case risk. Causal regression models, which regress the response on its direct causes, remain unchanged under arbitrary interventions on the covariates, but they are not always optimal in the above sense. For example, for linear models and bounded interventions, alternative solutions have been shown to be minimax prediction optimal. We introduce the formal framework of distribution generalization that allows us to analyze the above problem in partially observed nonlinear models for both direct interventions on $X$ and interventions that occur indirectly via exogenous variables $A$ . It takes into account that, in practice, minimax solutions need to be identified from data. Our framework allows us to characterize under which class of interventions the causal function is minimax optimal. We prove sufficient conditions for distribution generalization and present corresponding impossibility results. We propose a practical method, NILE, that achieves distribution generalization in a nonlinear IV setting with linear extrapolation. We prove consistency and present empirical results.
31 January 2022 in Journal of the American Statistical Association
How does armed conflict influence tropical forest loss? For Colombia, both enhancing and reducing effect estimates have been reported. However, a lack of causal methodology has prevented establishing clear causal links between these two variables. In this work, we propose a class of causal models for spatio-temporal stochastic processes which allows us to formally define and quantify the causal effect of a vector of covariates X on a real-valued response Y. We introduce a procedure for estimating causal effects and a nonparametric hypothesis test for these effects being zero. Our application is based on geospatial information on conflict events and remote-sensing-based data on forest loss between 2000 and 2018 in Colombia. Across the entire country, we estimate the effect to be slightly negative (conflict reduces forest loss) but insignificant (P = 0.578), while at the provincial level, we find both positive effects (e.g., La Guajira, P = 0.047) and negative effects (e.g., Magdalena, P = 0.004). The proposed methods do not make strong distributional assumptions, and allow for arbitrarily many latent confounders, given that these confounders do not vary across time. Our theoretical findings are supported by simulations, and code is available online.
Sun-Induced fluorescence at 760 nm (F760) is increasingly being used to predict gross primary production (GPP) through light use efficiency (LUE) modeling, even though the mechanistic processes that link the two are not well understood. We analyzed the effect of nitrogen (N) and phosphorous (P) availability on the processes that link GPP and F760 in a Mediterranean grassland manipulated with nutrient addition. To do so, we used a combination of process-based modeling with Soil-Canopy Observation of Photosynthesis and Energy (SCOPE), and statistical analyses such as path modeling. With this study, we uncover the mechanisms that link the fertilization-driven changes in canopy nitrogen concentration (N%) to the observed changes in F760 and GPP. N addition changed plant community structure and increased canopy chlorophyll content, which jointly led to changes in photosynthetic active radiation (APAR), ultimately affecting both GPP and F760. Changes in the abundance of graminoids, (%graminoids) driven by N addition led to changes in structural properties of the canopy such as leaf angle distribution, that ultimately influenced observed F760 by controlling the escape probability of F760 (Fesc). In particular, we found a change in GPP–F760 relationship between the first and the second year of the experiment that was largely driven by the effect of plant type composition on Fesc, whose best predictor is %graminoids. The P addition led to a statistically significant increase on light use efficiency of fluorescence emission (LUEf), in particular in plots also with N addition, consistent with leaf level studies. The N addition induced changes in the biophysical properties of the canopy that led to a trade-off between surface temperature (Ts), which decreased, and F760 at leaf scale (F760leaf,fw), which increased. We found that Ts is an important predictor of the light use efficiency of photosynthesis, indicating the importance of Ts in LUE modeling approaches to predict GPP.
We consider the problem of predicting a response $Y$ from a set of covariates $X$ when test and training distributions differ. Since such differences may have causal explanations, we consider test distributions that emerge from interventions in a structural causal model, and focus on minimizing the worst-case risk. Causal regression models, which regress the response on its direct causes, remain unchanged under arbitrary interventions on the covariates, but they are not always optimal in the above sense. For example, for linear models and bounded interventions, alternative solutions have been shown to be minimax prediction optimal. We introduce the formal framework of distribution generalization that allows us to analyze the above problem in partially observed nonlinear models for both direct interventions on $X$ and interventions that occur indirectly via exogenous variables $A$. It takes into account that, in practice, minimax solutions need to be identified from data. Our framework allows us to characterize under which class of interventions the causal function is minimax optimal. We prove sufficient conditions for distribution generalization and present corresponding impossibility results. We propose a practical method, NILE, that achieves distribution generalization in a nonlinear IV setting with linear extrapolation. We prove consistency and present empirical results.
In many data scientific problems, we are interested in inferring causal relationships in the data generating mechanism. Here, we consider the following real-world question: how has the Colombian conflict influenced tropical forest loss? There is evidence for both enhancing and reducing impacts. Answering such questions requires the use of causal models. In this work, we propose a class of causal models for spatio-temporal stochastic processes. It allows us to formally define and quantify the causal effect of a vector of covariates $X$ on a real-valued response $Y$, even if the causal background knowledge is incomplete. We introduce a procedure for estimating causal effects, and a non-parametric hypothesis test for these effects being zero. The proposed methods do not make strong distributional assumptions, and allow for arbitrarily many latent confounders, given that these confounders do not vary across time (or, alternatively, they do not vary across space). When applying our causal methodology to the problem of conflict and forest loss, using data from 2000 to 2018, we find a reducing but insignificant causal effect of conflict on forest loss. Regionally, both enhancing and reducing effects can be identified. Our theoretical findings are supported by simulations, and code is available online.