Generalized Information Bottleneck for Gaussian Variables

The information bottleneck (IB) method offers an attractive framework for understanding representation learning, however its applications are often limited by its computational intractability. Analytical characterization of the IB method is not only of practical interest, but it can also lead to new insights into learning phenomena. Here we consider a generalized IB problem, in which the mutual information in the original IB method is replaced by correlation measures based on Renyi and Jeffreys divergences. We derive an exact analytical IB solution for the case of Gaussian correlated variables. Our analysis reveals a series of structural transitions, similar to those previously observed in the original IB case. We find further that although solving the original, Renyi and Jeffreys IB problems yields different representations in general, the structural transitions occur at the same critical tradeoff parameters, and the Renyi and Jeffreys IB solutions perform well under the original IB objective. Our results suggest that formulating the IB method with alternative correlation measures could offer a strategy for obtaining an approximate solution to the original IB problem.Comment: 7 pages, 3 figure

Catalysis in Quantum Information Theory

Catalysts open up new reaction pathways which can speed up chemical reactions while not consuming the catalyst. A similar phenomenon has been discovered in quantum information science, where physical transformations become possible by utilizing a (quantum) degree of freedom that remains unchanged throughout the process. In this review, we present a comprehensive overview of the concept of catalysis in quantum information science and discuss its applications in various physical contexts.Comment: Review paper; Comments and suggestions welcome

Quantum hypothesis testing in many-body systems

Peer reviewe

General properties of holographic entanglement entropy

The Ryu-Takayanagi formula implies many general properties of entanglement entropies in holographic theories. We review the known properties, such as continuity, strong subadditivity, and monogamy of mutual information, and fill in gaps in some of the previously-published proofs. We also add a few new properties, including: properties of the map from boundary regions to bulk regions implied by the RT formula, such as monotonicity; conditions under which subadditivity-type inequalities are saturated; and an inequality concerning reflection-symmetric states. We attempt to draw lessons from these properties about the structure of the reduced density matrix in holographic theories.Comment: 27 page

Entropic Continuity Bounds & Eventually Entanglement-Breaking Channels

This thesis combines two parallel research directions: an exploration into the continuity properties of certain entropic quantities, and an investigation into a simple class of physical systems whose time evolution is given by the repeated application of a quantum channel. In the first part of the thesis, we present a general technique for establishing local and uniform continuity bounds for Schur concave functions; that is, for real-valued functions which are decreasing in the majorization pre-order. Continuity bounds provide a quantitative measure of robustness, addressing the following question: If there is some uncertainty or error in the input, how much uncertainty is there in the output? Our technique uses a particular relationship between majorization and the trace distance between quantum states (or total variation distance, in the case of probability distributions). Namely, the majorization pre-order attains a maximum and a minimum over ε-balls in this distance. By tracing the path of the majorization-minimizer as a function of the distance ε, we obtain the path of ``majorization flow’’. An analysis of the derivatives of Schur concave functions along this path immediately yields tight continuity bounds for such functions. In this way, we find a new proof of the Audenaert-Fannes continuity bound for the von Neumann entropy, and the necessary and sufficient conditions for its saturation, in a universal framework which extends to the other functions, including the Rényi and Tsallis entropies. In particular, we prove a novel uniform continuity bound for the α-Rényi entropy with α > 1 with much improved dependence on the dimension of the underlying system and the parameter α compared to previously known bounds. We show that this framework can also be used to provide continuity bounds for other Schur concave functions, such as the number of connected components of a certain random graph model as a function of the underlying probability distribution, and the number of distinct realizations of a random variable in some fixed number of independent trials as a function of the underlying probability mass function. The former has been used in modeling the spread of epidemics, while the latter has been studied in the context of estimating measures of biodiversity from observations; in these contexts, our continuity bounds provide quantitative estimates of robustness to noise or data collection errors. In the second part, we consider repeated interaction systems, in which a system of interest interacts with a sequence of probes, i.e. environmental systems, one at a time. The state of the system after each interaction is related to the state of the system before the interaction by the so-called reduced dynamics, which is described by the action of a quantum channel. When each probe and the way it interacts with the system is identical, the reduced dynamics at each step is identical. In this scenario, under the additional assumption that the reduced dynamics satisfies a faithfulness property, we characterize which repeated interaction systems break any initially-present entanglement between the system and an untouched reference, after finitely many steps. In this case, the reduced dynamics is said to be eventually entanglement-breaking. This investigation helps improve our understanding of which kinds of noisy time evolution destroy entanglement. When the probes and their interactions with the system are slowly-varying (i.e. adiabatic), we analyze the saturation of Landauer's bound, an inequality between the entropy change of the system and the energy change of the probes, in the limit in which the number of steps tends to infinity and both the difference between consecutive probes and the difference between their interactions vanishes. This analysis proceeds at a fine-grained level by means of a two-time measurement protocol, in which the energy of the probes is measured before and after each interaction. The quantities of interest are then studied as random variables on the space of outcomes of the energy measurements of the probes, providing a deeper insight into the interrelations between energy and entropy in this setting.Cantab Capital Institute for the Mathematics of Informatio

Generalizations of Fano's Inequality for Conditional Information Measures via Majorization Theory

Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon and R\'{e}nyi. When specialized to these measures, it recovers and generalizes the classical inequalities. Key to the derivation is the construction of an appropriate conditional distribution inducing a desired marginal distribution on a countably infinite alphabet. The construction is based on the infinite-dimensional version of Birkhoff's theorem proven by R\'{e}v\'{e}sz [Acta Math. Hungar. 1962, 3, 188{\textendash}198], and the constraint of maintaining a desired marginal distribution is similar to coupling in probability theory. Using our Fano-type inequalities for Shannon's and R\'{e}nyi's information measures, we also investigate the asymptotic behavior of the sequence of Shannon's and R\'{e}nyi's equivocations when the error probabilities vanish. This asymptotic behavior provides a novel characterization of the asymptotic equipartition property (AEP) via Fano's inequality.Comment: 44 pages, 3 figure