We propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y, Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which suggests that shared information and unique information should depend only on the marginal distributions of the pairs (X, Y) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.
1 September 2010 in Advances in Applied Mathematics
Recently, a series of papers addressed the problem of decomposing the information of two random variables into shared information, unique information and synergistic information. Several measures were proposed, although still no consensus has been reached. Here, we compare these proposals with an older approach to define synergistic information based on the projections on exponential families containing only up to k-th order interactions. We show that these measures are not compatible with a decomposition into unique, shared and synergistic information if one requires that all terms are always non-negative (local positivity). We illustrate the difference between the two measures for multivariate Gaussians.
We consider determinantal ideals, where the generating minors are encoded in a hypergraph. We study when the generating minors form a Gröbner basis. In this case, the ideal is radical, and we can describe algebraic and numerical invariants of these ideals in terms of combinatorial data of their hypergraphs, such as the clique decomposition. In particular, we can construct a minimal free resolution as a tensor product of the minimal free resolution of their cliques. For several classes of hypergraphs we find a combinatorial description of the minimal primes in terms of a prime splitting. That is, we write the determinantal ideal as a sum of smaller determinantal ideals such that each minimal prime is a sum of minimal primes of the summands.