Information inequalities and Generalized Graph Entropies

In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy have been considered for this study. Our main results involve establishing formal relationship, in the form of implicit inequalities, between these two kinds of measures when defined for graphs. Further, we also state and prove inequalities connecting the classical partition-based graph entropies and the functional-based entropy measures. In addition, several explicit inequalities are derived for special classes of graphs.Comment: A preliminary version. To be submitted to a journa

Approximate entropy of network parameters

We study the notion of approximate entropy within the framework of network theory. Approximate entropy is an uncertainty measure originally proposed in the context of dynamical systems and time series. We firstly define a purely structural entropy obtained by computing the approximate entropy of the so called slide sequence. This is a surrogate of the degree sequence and it is suggested by the frequency partition of a graph. We examine this quantity for standard scale-free and Erd\H{o}s-R\'enyi networks. By using classical results of Pincus, we show that our entropy measure converges with network size to a certain binary Shannon entropy. On a second step, with specific attention to networks generated by dynamical processes, we investigate approximate entropy of horizontal visibility graphs. Visibility graphs permit to naturally associate to a network the notion of temporal correlations, therefore providing the measure a dynamical garment. We show that approximate entropy distinguishes visibility graphs generated by processes with different complexity. The result probes to a greater extent these networks for the study of dynamical systems. Applications to certain biological data arising in cancer genomics are finally considered in the light of both approaches.Comment: 11 pages, 5 EPS figure

Entropy of Overcomplete Kernel Dictionaries

In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the orthogonality condition of the atoms, yielding overcomplete dictionaries with an extended number of atoms. More generally than the linear decomposition, overcomplete kernel dictionaries provide an elegant nonlinear extension by defining the atoms through a mapping kernel function (e.g., the gaussian kernel). Models based on such kernel dictionaries are used in neural networks, gaussian processes and online learning with kernels. The quality of an overcomplete dictionary is evaluated with a diversity measure the distance, the approximation, the coherence and the Babel measures. In this paper, we develop a framework to examine overcomplete kernel dictionaries with the entropy from information theory. Indeed, a higher value of the entropy is associated to a further uniform spread of the atoms over the space. For each of the aforementioned diversity measures, we derive lower bounds on the entropy. Several definitions of the entropy are examined, with an extensive analysis in both the input space and the mapped feature space.Comment: 10 page

Entanglement, quantum randomness, and complexity beyond scrambling

Scrambling is a process by which the state of a quantum system is effectively randomized due to the global entanglement that "hides" initially localized quantum information. In this work, we lay the mathematical foundations of studying randomness complexities beyond scrambling by entanglement properties. We do so by analyzing the generalized (in particular R\'enyi) entanglement entropies of designs, i.e. ensembles of unitary channels or pure states that mimic the uniformly random distribution (given by the Haar measure) up to certain moments. A main collective conclusion is that the R\'enyi entanglement entropies averaged over designs of the same order are almost maximal. This links the orders of entropy and design, and therefore suggests R\'enyi entanglement entropies as diagnostics of the randomness complexity of corresponding designs. Such complexities form a hierarchy between information scrambling and Haar randomness. As a strong separation result, we prove the existence of (state) 2-designs such that the R\'enyi entanglement entropies of higher orders can be bounded away from the maximum. However, we also show that the min entanglement entropy is maximized by designs of order only logarithmic in the dimension of the system. In other words, logarithmic-designs already achieve the complexity of Haar in terms of entanglement, which we also call max-scrambling. This result leads to a generalization of the fast scrambling conjecture, that max-scrambling can be achieved by physical dynamics in time roughly linear in the number of degrees of freedom.Comment: 72 pages, 4 figures. Rewritten version with new title. v3: published versio

Area laws for the entanglement entropy - a review

Physical interactions in quantum many-body systems are typically local: Individual constituents interact mainly with their few nearest neighbors. This locality of interactions is inherited by a decay of correlation functions, but also reflected by scaling laws of a quite profound quantity: The entanglement entropy of ground states. This entropy of the reduced state of a subregion often merely grows like the boundary area of the subregion, and not like its volume, in sharp contrast with an expected extensive behavior. Such "area laws" for the entanglement entropy and related quantities have received considerable attention in recent years. They emerge in several seemingly unrelated fields, in the context of black hole physics, quantum information science, and quantum many-body physics where they have important implications on the numerical simulation of lattice models. In this Colloquium we review the current status of area laws in these fields. Center stage is taken by rigorous results on lattice models in one and higher spatial dimensions. The differences and similarities between bosonic and fermionic models are stressed, area laws are related to the velocity of information propagation, and disordered systems, non-equilibrium situations, classical correlation concepts, and topological entanglement entropies are discussed. A significant proportion of the article is devoted to the quantitative connection between the entanglement content of states and the possibility of their efficient numerical simulation. We discuss matrix-product states, higher-dimensional analogues, and states from entanglement renormalization and conclude by highlighting the implications of area laws on quantifying the effective degrees of freedom that need to be considered in simulations.Comment: 28 pages, 2 figures, final versio