4,544 research outputs found
Partial information decomposition as a unified approach to the specification of neural goal functions
In many neural systems anatomical motifs are present repeatedly, but despite their structural similarity they can serve very different tasks. A prime example for such a motif is the canonical microcircuit of six-layered neo-cortex, which is repeated across cortical areas, and is involved in a number of different tasks (e.g. sensory, cognitive, or motor tasks). This observation has spawned interest in finding a common underlying principle, a ‘goal function’, of information processing implemented in this structure. By definition such a goal function, if universal, cannot be cast in processing-domain specific language (e.g. ‘edge filtering’, ‘working memory’). Thus, to formulate such a principle, we have to use a domain-independent framework. Information theory offers such a framework. However, while the classical framework of information theory focuses on the relation between one input and one output (Shannon’s mutual information), we argue that neural information processing crucially depends on the combination of multiple inputs to create the output of a processor. To account for this, we use a very recent extension of Shannon Information theory, called partial information decomposition (PID). PID allows to quantify the information that several inputs provide individually (unique information), redundantly (shared information) or only jointly (synergistic information) about the output. First, we review the framework of PID. Then we apply it to reevaluate and analyze several earlier proposals of information theoretic neural goal functions (predictive coding, infomax and coherent infomax, efficient coding). We find that PID allows to compare these goal functions in a common framework, and also provides a versatile approach to design new goal functions from first principles. Building on this, we design and analyze a novel goal function, called ‘coding with synergy’, which builds on combining external input and prior knowledge in a synergistic manner. We suggest that this novel goal function may be highly useful in neural information processing
Entropy of Overcomplete Kernel Dictionaries
In signal analysis and synthesis, linear approximation theory considers a
linear decomposition of any given signal in a set of atoms, collected into a
so-called dictionary. Relevant sparse representations are obtained by relaxing
the orthogonality condition of the atoms, yielding overcomplete dictionaries
with an extended number of atoms. More generally than the linear decomposition,
overcomplete kernel dictionaries provide an elegant nonlinear extension by
defining the atoms through a mapping kernel function (e.g., the gaussian
kernel). Models based on such kernel dictionaries are used in neural networks,
gaussian processes and online learning with kernels.
The quality of an overcomplete dictionary is evaluated with a diversity
measure the distance, the approximation, the coherence and the Babel measures.
In this paper, we develop a framework to examine overcomplete kernel
dictionaries with the entropy from information theory. Indeed, a higher value
of the entropy is associated to a further uniform spread of the atoms over the
space. For each of the aforementioned diversity measures, we derive lower
bounds on the entropy. Several definitions of the entropy are examined, with an
extensive analysis in both the input space and the mapped feature space.Comment: 10 page
The role of quantum information in thermodynamics --- a topical review
This topical review article gives an overview of the interplay between
quantum information theory and thermodynamics of quantum systems. We focus on
several trending topics including the foundations of statistical mechanics,
resource theories, entanglement in thermodynamic settings, fluctuation theorems
and thermal machines. This is not a comprehensive review of the diverse field
of quantum thermodynamics; rather, it is a convenient entry point for the
thermo-curious information theorist. Furthermore this review should facilitate
the unification and understanding of different interdisciplinary approaches
emerging in research groups around the world.Comment: published version. 34 pages, 6 figure
Composable security proof for continuous-variable quantum key distribution with coherent states
We give the first composable security proof for continuous-variable quantum
key distribution with coherent states against collective attacks. Crucially, in
the limit of large blocks the secret key rate converges to the usual value
computed from the Holevo bound. Combining our proof with either the de Finetti
theorem or the Postselection technique then shows the security of the protocol
against general attacks, thereby confirming the long-standing conjecture that
Gaussian attacks are optimal asymptotically in the composable security
framework.
We expect that our parameter estimation procedure, which does not rely on any
assumption, will find applications elsewhere, for instance for the reliable
quantification of continuous-variable entanglement in finite-size settings.Comment: 27 pages, 1 figure. v2: added a version of the AEP valid for
conditional state
- …