The formulation of the Partial Information Decomposition (PID) framework by Williams and Beer in 2010 attracted a significant amount of attention to the problem of defining redundant (or shared), unique and synergistic (or complementary) components of mutual information that a set of source variables provides about a target. This attention resulted in a number of measures proposed to capture these concepts, theoretical investigations into such measures, and applications to empirical data (in particular to datasets from neuroscience). In this Special Issue on “Information Decomposition of Target Effects from Multi-Source Interactions” at Entropy, we have gathered current work on such information decomposition approaches from many of the leading research groups in the field. We begin our editorial by providing the reader with a review of previous information decomposition research, including an overview of the variety of measures proposed, how they have been interpreted and applied to empirical investigations. We then introduce the articles included in the special issue one by one, providing a similar categorisation of these articles into: i. proposals of new measures; ii. theoretical investigations into properties and interpretations of such approaches, and iii. applications of these measures in empirical studies. We finish by providing an outlook on the future of the field.

Recently, a series of papers addressed the problem of decomposing the information of two random variables into shared information, unique information and synergistic information. Several measures were proposed, although still no consensus has been reached. Here, we compare these proposals with an older approach to define synergistic information based on the projections on exponential families containing only up to k-th order interactions. We show that these measures are not compatible with a decomposition into unique, shared and synergistic information if one requires that all terms are always non-negative (local positivity). We illustrate the difference between the two measures for multivariate Gaussians.

Suppose we have a pair of information channels, ${\kappa }_1,{\kappa }_2$ , with a common input. The Blackwell order is a partial order over channels that compares ${\kappa }_1$ and ${\kappa }_2$ by the maximal expected utility an agent can obtain when decisions are based on the channel outputs. Equivalently, ${\kappa }_1$ is said to be Blackwell-inferior to ${\kappa }_2$ if and only if ${\kappa }_1$ can be constructed by garbling the output of ${\kappa }_2$ . A related partial order stipulates that ${\kappa }_2$ is more capable than ${\kappa }_1$ if the mutual information between the input and output is larger for ${\kappa }_2$ than for ${\kappa }_1$ for any distribution over inputs. A Blackwell-inferior channel is necessarily less capable. However, examples are known where ${\kappa }_1$ is less capable than ${\kappa }_2$ but not Blackwell-inferior. We show that this may even happen when ${\kappa }_1$ is constructed by coarse-graining the inputs of ${\kappa }_2$ . Such a coarse-graining is a special kind of “pre-garbling” of the channel inputs. This example directly establishes that the expected value of the shared utility function for the coarse-grained channel is larger than it is for the non-coarse-grained channel. This contradicts the intuition that coarse-graining can only destroy information and lead to inferior channels. We also discuss our results in the context of information decompositions.

We consider the problem of quantifying the information shared by a pair of random variables $X_1,X_2$ about another variable S. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about S is bounded from below by the information shared about $f\left(S\right)$ for any function f. We show that our measure leads to a new nonnegative decomposition of the mutual information $I(S;X_1{X}_2)$ into shared, complementary and unique components. We study properties of this decomposition and show that a left monotonic shared information is not compatible with a Blackwell interpretation of unique information. We also discuss whether it is possible to have a decomposition in which both shared and unique information are left monotonic.

The statistical description and modeling of volatility plays a prominent role in econometrics, risk management and finance. GARCH and stochastic volatility models have been extensively studied and are routinely fitted to market data, albeit providing a phenomenological description only. In contrast, agent-based modeling starts from the premise that modern economies consist of a vast number of individual actors with heterogeneous expectations and incentives. Observed market statistics then emerge from the collective dynamics of many actors following heterogeneous, yet simple rules. On the one hand, such models generate volatility dynamics, qualitatively matching several stylized facts. On the other hand, they illustrate the possible role of different mechanisms, such as chartist trading and herding behavior. Yet, rigorous and quantitative statistical fits are still mostly lacking. Here, we propose Hamiltonian Monte Carlo, an efficient and scalable Markov chain Monte Carlo algorithm, as a general method for Bayesian inference of agent-based models. In particular, we implement several models by Vikram and Sinha, Franke and Westerhoff and Alfarano, Lux and Wagner in Stan, an accessible probabilistic programming language for Bayesian modeling. We also compare the performance of these models with standard econometric models of the GARCH and stochastic volatility families. We find that the best agent-based models are on par with stochastic volatility models in terms of predictive likelihood, yet exhibit challenging posterior geometries requiring care in model comparison and sophisticated sampling algorithms.