We introduce a gain function viewpoint of information leakage by proposing maximal $g$ -leakage, a rich class of operationally meaningful leakage measures that subsumes recently introduced leakage measures — maximal leakage and maximal $\alpha $ -leakage. In maximal $g$ -leakage, the gain of an adversary in guessing an unknown random variable is measured using a gain function applied to the probability of correctly guessing. In particular, maximal $g$ -leakage captures the multiplicative increase, upon observing $Y$ , in the expected gain of an adversary in guessing a randomized function of $X$ , maximized over all such randomized functions. We also consider the scenario where an adversary can make multiple attempts to guess the randomized function of interest. We show that maximal leakage is an upper bound on maximal $g$ -leakage under multiple guesses, for any non-negative gain function $g$ . We obtain a closed-form expression for maximal $g$ -leakage under multiple guesses for a class of concave gain functions. We also study maximal $g$ -leakage measure for a specific class of gain functions related to the $\alpha $ -loss, that interpolates log-loss ( $\alpha =1$ ) and (soft) 0–1 loss ( $\alpha =\infty $ ). In particular, we first completely characterize the minimal expected $\alpha $ -loss under multiple guesses and analyze how the corresponding leakage measure is affected with the number of guesses. We show that a new measure of divergence that belongs to the class of Bregman divergences captures the relative performance of an arbitrary adversarial strategy with respect to an optimal strategy in minimizing the expected $\alpha $ -loss. Finally, we study two variants of maximal $g$ -leakage depending on the type of adversary and obtain closed-form expressions for them, which do not depend on the particular gain function considered as long as it satisfies some mild regularity conditions. We do this by developing a variational characterization for the Rényi divergence of order infinity which naturally generalizes the definition of pointwise maximal leakage to incorporate arbitrary gain functions.

Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and f-GANs which minimize f-divergences. We also show that all symmetric f-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to α-GANs, defined using α-loss, a tunable CPE loss family parametrized by α(0,∞. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player’s objective using α-loss to obtain (αD,αG)-GANs. We show that the resulting non-zero sum game simplifies to minimizing an f-divergence under appropriate conditions on (αD,αG). Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning (αD,αG) in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.

We consider interactive computation of randomized functions between two users with the following privacy requirement: the interaction should not reveal to either user any extra information about the other user’s input and output other than what can be inferred from the user’s own input and output. We also consider the case where privacy is required against only one of the users. For both cases, we give single-letter expressions for feasibility and optimal rates of communication. Then we discuss the role of common randomness and interaction in both privacy settings. We also study perfectly secure non-interactive computation when only one of the users computes a randomized function based on a single transmission from the other user. We characterize randomized functions which can be perfectly securely computed in this model and obtain tight bounds on the optimal message lengths in all the privacy settings.

We study the problem of simulating a two-user multiple-access channel (MAC) over a multiple access network of noiseless links. Two encoders observe independent and identically distributed (i.i.d.) copies of a source random variable each, while a decoder observes i.i.d. copies of a side-information random variable. There are rate-limited noiseless communication links between each encoder and the decoder, and there is independent pairwise shared randomness between all the three possible pairs of nodes. The decoder has to output approximately i.i.d. copies of another random variable jointly distributed with the two sources and the side information. We are interested in the rate tuples which permit this simulation. This setting can be thought of as a multi-terminal generalization of the point-to-point channel simulation problem studied by Bennett et al. (2002) and Cuff (2013). When the pairwise shared randomness between the encoders is absent, the setting reduces to a special case of MAC simulation using another MAC studied by Haddadpour et al. (2013). We establish that the presence of encoder shared randomness can strictly improve the communication rate requirements. We first show that the inner bound derived from Haddadpour et al. (2013) is tight when the sources at the encoders are conditionally independent given the side-information at the decoder. This result recovers the existing results on point-to-point channel simulation and function computation over such multi-terminal networks. We then explicitly compute the communication rate regions for an example both with and without the encoder shared randomness and demonstrate that its presence strictly reduces the communication rates. Inner and outer bounds for the general case are also obtained.

We introduce a family of information leakage measures called maximal $(\alpha,\beta)$ -leakage ( $\text{M}\alpha $ beL), parameterized by real numbers $\alpha $ and $\beta $ greater than or equal to 1. The measure is formalized via an operational definition involving an adversary guessing an unknown (randomized) function of the data given the released data. We obtain a simplified computable expression for the measure and show that it satisfies several basic properties such as monotonicity in $\beta $ for a fixed $\alpha $ , non-negativity, data processing inequalities, and additivity over independent releases. We highlight the relevance of this family by showing that it bridges several known leakage measures, including maximal $\alpha $ -leakage $(\beta =1)$ , maximal leakage $(\alpha =\infty,\beta =1)$ , local differential privacy (LDP) $(\alpha =\infty,\beta =\infty)$ , and local Rényi differential privacy (LRDP) $(\alpha =\beta)$ , thereby giving an operational interpretation to local Rényi differential privacy. We also study a conditional version of $\text{M}\alpha $ beL on leveraging which we recover differential privacy and Rényi differential privacy. A new variant of LRDP, which we call maximal Rényi leakage, appears as a special case of $\text{M}\alpha $ beL for $\alpha =\infty $ that smoothly tunes between maximal leakage ( $\beta =1$ ) and LDP ( $\beta =\infty $ ). Finally, we show that a vector form of the maximal Rényi leakage relaxes differential privacy under Gaussian and Laplacian mechanisms.

We prove a two-way correspondence between the min-max optimization of general CPE loss function GANs and the minimization of associated f-divergences. We then focus on α-GAN, defined via the α-loss, which interpolates several GANs (Hellinger, vanilla, Total Variation) and corresponds to the minimization of the Arimoto divergence. We show that the Arimoto divergences induced by α-GAN equivalently converge, for all α∈ℝ>0∪{∞}. However, under restricted learning models and finite samples, we provide estimation bounds which indicate diverse GAN behavior as a function of α. Finally, we present empirical results on a toy dataset that highlight the practical utility of tuning the α hyperparameter.