Given two random variables X and Y , an operational approach is undertaken to quantify the “leakage” of information from X to Y. The resulting measure L(X→Y ) is called maximal leakage, and is defined as the multiplicative increase, upon observing Y , of the probability of correctly guessing a randomized function of X, maximized over all such randomized functions. A closedform expression for L(X→Y ) is given for discrete X and Y, and it is subsequently generalized to handle a large class of random variables. The resulting properties are shown to be consistent with an axiomatic view of a leakage measure, and the definition is shown to be robust to variations in the setup. Moreover, a variant of the Shannon cipher system is studied, in which performance of an encryption scheme is measured using maximal leakage. A singleletter characterization of the optimal limit of (normalized) maximal leakage is derived and asymptotically-optimal encryption schemes are demonstrated. Furthermore, the sample complexity of estimating maximal leakage from data is characterized up to subpolynomial factors. Finally, the guessing framework used to define maximal leakage is used to give operational interpretations of commonly used leakage measures, such as Shannon capacity, maximal correlation, and local differential privacy.

The entropy of a quantum system is a measure of its randomness, and has applications in measuring quantum entanglement. We study the problem of estimating the von Neumann entropy, S(ρ), and Rényi entropy, Sα(ρ) of an unknown mixed quantum state ρ in d dimensions, given access to independent copies of ρ. We provide algorithms with copy complexity O(d2/α) for estimating Sα(ρ) for α1. These bounds are at least quadratic in d, which is the order dependence on the number of copies required for estimating the entire state ρ. For integral α>1, on the other hand, we provide an algorithm for estimating Sα(ρ) with a sub-quadratic copy complexity of O(d2-2/α), and we show the optimality of the algorithms by providing a matching lower bound.

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. These results find applications in adaptive data analysis, where multiple dependencies are introduced and in learning theory, where they can be employed to bound the generalization error of a learning algorithm. Bounds are given in terms of Sibson's Mutual Information, α-Divergences, Hellinger Divergences, and f-Divergences. A case of particular interest is the Maximal Leakage (or Sibson's Mutual Information of order infinity), since this measure is robust to post-processing and composes adaptively. The corresponding bound can be seen as a generalization of classical bounds, such as Hoeffding's and McDiarmid's inequalities, to the case of dependent random variables.

The secrecy of a communication system in which both the legitimate receiver and an eavesdropper are allowed some distortion is investigated. The secrecy metric considered is the exponent of the probability that the eavesdropper estimates the source sequence successfully within an acceptable distortion level. The problem is first studied when the transmitter and the legitimate receiver do not share any key and the transmitter is not subject to a rate constraint, which corresponds to a stylized model of a side channel and reveals connections to source coding with side information. The setting is then generalized to include a shared secret key between the transmitter and the legitimate receiver and a rate constraint on the transmitter, which corresponds to the Shannon cipher system. A single-letter characterization of the highest achievable exponent is provided, and asymptotically optimal strategies for both the primary user and the eavesdropper are demonstrated.

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f−divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals. This allows us to control the measure of an event under the joint, using the product of the marginals (typically easier to compute) and a measure of how much the two distributions differ, i.e., an f−divergence between the joint and the product of the marginals, also known in the literature as f−Mutual Information. The result is general enough to induce, as special cases, bounds involving χ ² -divergence, Hellinger distance, Total Variation, etc. Moreover, it also recovers a result involving Rényi’s α−divergence. As an application, we provide bounds on the generalization error of learning algorithms via f−divergences.

A variation of the Shannon cipher system, in which lossy communication is allowed and performance of an encryption scheme is measured in terms of maximal leakage (recently proposed by the authors [1]), is investigated. The asymptotic behavior of normalized maximal leakage is studied, and a single-letter characterization of the optimal limit is derived. Moreover, asymptotically-optimal encryption schemes are demonstrated.