Addressing GAN Training Instabilities via Tunable Classification Losses

Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and $f$-GANs which minimize $f$-divergences. We also show that all symmetric $f$-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to $\alpha$-GANs, defined using $\alpha$-loss, a tunable CPE loss family parametrized by $\alpha\in(0,\infty]$. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player's objective using $\alpha$-loss to obtain $(\alpha_D,\alpha_G)$-GANs. We show that the resulting non-zero sum game simplifies to minimizing an $f$-divergence under appropriate conditions on $(\alpha_D,\alpha_G)$. Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning $(\alpha_D,\alpha_G)$ in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.Comment: arXiv admin note: text overlap with arXiv:2302.1432

An Alphabet of Leakage Measures

We introduce a family of information leakage measures called maximal $\alpha,\beta$-leakage, parameterized by real numbers $\alpha$ and $\beta$. The measure is formalized via an operational definition involving an adversary guessing an unknown function of the data given the released data. We obtain a simple, computable expression for the measure and show that it satisfies several basic properties such as monotonicity in $\beta$ for a fixed $\alpha$, non-negativity, data processing inequalities, and additivity over independent releases. Finally, we highlight the relevance of this family by showing that it bridges several known leakage measures, including maximal $\alpha$-leakage $(\beta=1)$, maximal leakage $(\alpha=\infty,\beta=1)$, local differential privacy $(\alpha=\infty,\beta=\infty)$, and local Renyi differential privacy $(\alpha=\beta)$

The Role of Interaction and Common Randomness in Two-User Secure Computation

This paper has been presented at : 2018 IEEE International Symposium On Information Theory (ISIT)We consider interactive computation of randomized functions between two users with the following privacy requirement: the interactive communication should not reveal to either user any extra information about the other user's input and output other than what can be inferred from the user's own input and output. We also consider the case where privacy is required against only one of the users. For both cases, we give single-letter expressions for feasibility and optimal rates of communication. Then we discuss the role of common randomness and interaction in both privacy settings.Gowtham R. Kurri was supported by a travel fellowship from the Sarojini Damodaran Foundation. This work was done while Jithin Ravi was at Tata Institute of Fundamental Research. He has received funding from ERC grant 714161

Interactive Secure Function Computation

We consider interactive computation of randomized functions between two users with the following privacy requirement: the interaction should not reveal to either user any extra information about the other user's input and output other than what can be inferred from the user's own input and output. We also consider the case where privacy is required against only one of the users. For both cases, we give single-letter expressions for feasibility and optimal rates of communication. Then we discuss the role of common randomness and interaction in both privacy settings. We also study perfectly secure non-interactive computation when only one of the users computes a randomized function based on a single transmission from the other user. We characterize randomized functions which can be perfectly securely computed in this model and obtain tight bounds on the optimal message lengths in all the privacy settings.Comment: 30 pages. Revised based on comments from the reviewer

Multiple Access Channel Simulation

We study the problem of simulating a two-user multiple access channel over a multiple access network of noiseless links. Two encoders observe independent and identically distributed (i.i.d.) copies of a source random variable each, while a decoder observes i.i.d. copies of a side-information random variable. There are rate-limited noiseless communication links and independent pairwise shared randomness resources between each encoder and the decoder. The decoder has to output approximately i.i.d. copies of another random variable jointly distributed with the two sources and the side information. We are interested in the rate tuples which permit this simulation. This setting can be thought of as a multi-terminal generalization of the point-to-point channel simulation problem studied by Bennett et al. (2002) and Cuff (2013). General inner and outer bounds on the rate region are derived. For the specific case where the sources at the encoders are conditionally independent given the side-information at the decoder, we completely characterize the rate region. Our bounds recover the existing results on function computation over such multi-terminal networks. We then show through an example that an additional independent source of shared randomness between the encoders strictly improves the communication rate requirements, even if the additional randomness is not available to the decoder. Furthermore, we provide inner and outer bounds for this more general setting with independent pairwise shared randomness resources between all the three possible node pairs.Comment: 33 pages, 3 figure