Learning, compression, and leakage: Minimising classification error via meta-universal compression principles

Learning and compression are driven by the common aim of identifying and exploiting statistical regularities in data, which opens the door for fertile collaboration between these areas. A promising group of compression techniques for learning scenarios is normalised maximum likelihood (NML) coding, which provides strong guarantees for compression of small datasets — in contrast with more popular estimators whose guarantees hold only in the asymptotic limit. Here we consider a NMLbased decision strategy for supervised classification problems, and show that it attains heuristic PAC learning when applied to a wide variety of models. Furthermore, we show that the misclassification rate of our method is upper bounded by the maximal leakage, a recently proposed metric to quantify the potential of data leakage in privacy-sensitive scenarios

Deep unsupervised clustering with Gaussian mixture variational autoencoders

We study a variant of the variational autoencoder model with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep generative models. We observe that the standard variational approach in these models is unsuited for unsupervised clustering, and mitigate this problem by leveraging a principled information-theoretic regularisation term known as consistency violation. Adding this term to the standard variational optimisation objective yields networks with both meaningful internal representations and well-defined clusters. We demonstrate the performance of this scheme on synthetic data, MNIST and SVHN, showing that the obtained clusters are distinct, interpretable and result in achieving higher performance on unsupervised clustering classification than previous approaches

An operational information decomposition via synergistic disclosure

Abstract Multivariate information decompositions hold promise to yield insight into complex systems, and stand out for their ability to identify synergistic phenomena. However, the adoption of these approaches has been hindered by there being multiple possible decompositions, and no precise guidance for preferring one over the others. At the heart of this disagreement lies the absence of a clear operational interpretation of what synergistic information is. Here we fill this gap by proposing a new information decomposition based on a novel operationalisation of informational synergy, which leverages recent developments in the literature of data privacy. Our decomposition is defined for any number of information sources, and its atoms can be calculated using elementary optimisation techniques. The decomposition provides a natural coarse-graining that scales gracefully with the system’s size, and is applicable in a wide range of scenarios of practical interest.</jats:p

The strength of weak integrated information theory

The integrated information theory of consciousness (IIT) is divisive: while some believe it provides an unprecedentedly powerful approach to address the ‘hard problem’, others dismiss it on grounds that it is untestable. We argue that the appeal and applicability of IIT can be greatly widened if we distinguish two flavours of the theory: strong IIT, which identifies consciousness with specific properties associated with maxima of integrated information; and weak IIT, which tests pragmatic hypotheses relating aspects of consciousness to broader measures of information dynamics. We review challenges for strong IIT, explain how existing empirical findings are well explained by weak IIT without needing to commit to the entirety of strong IIT, and discuss the outlook for both flavours of IIT

Greater than the parts: a review of the information decomposition approach to causal emergence.

Emergence is a profound subject that straddles many scientific disciplines, including the formation of galaxies and how consciousness arises from the collective activity of neurons. Despite the broad interest that exists on this concept, the study of emergence has suffered from a lack of formalisms that could be used to guide discussions and advance theories. Here, we summarize, elaborate on, and extend a recent formal theory of causal emergence based on information decomposition, which is quantifiable and amenable to empirical testing. This theory relates emergence with information about a system's temporal evolution that cannot be obtained from the parts of the system separately. This article provides an accessible but rigorous introduction to the framework, discussing the merits of the approach in various scenarios of interest. We also discuss several interpretation issues and potential misunderstandings, while highlighting the distinctive benefits of this formalism. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'

Assessing awareness in severe Alzheimer’s disease

There is an urgent need to understand the nature of awareness in people with severe Alzheimer’s disease (AD) to ensure effective person-centered care. Objective biomarkers of awareness validated in other clinical groups (e.g., anesthesia, minimally conscious states) offer an opportunity to investigate awareness in people with severe AD. In this article we demonstrate the feasibility of using Transcranial magnetic stimulation (TMS) combined with EEG, event related potentials (ERPs) and fMRI to assess awareness in severe AD. TMS-EEG was performed in six healthy older controls and three people with severe AD. The perturbational complexity index (PCIST) was calculated as a measure of capacity for conscious awareness. People with severe AD demonstrated a PCIST around or below the threshold for consciousness, suggesting reduced capacity for consciousness. ERPs were recorded during a visual perception paradigm. In response to viewing faces, two patients with severe AD provisionally demonstrated similar visual awareness negativity to healthy controls. Using a validated fMRI movie-viewing task, independent component analysis in two healthy controls and one patient with severe AD revealed activation in auditory, visual and fronto-parietal networks. Activation patterns in fronto-parietal networks did not significantly correlate between the patient and controls, suggesting potential differences in conscious awareness and engagement with the movie. Although methodological issues remain, these results demonstrate the feasibility of using objective measures of awareness in severe AD. We raise a number of challenges and research questions that should be addressed using these biomarkers of awareness in future studies to improve understanding and care for people with severe AD

Psychedelics and schizophrenia: Distinct alterations to Bayesian inference

Schizophrenia and states induced by certain psychotomimetic drugs may share some physiological and phenomenological properties, but they differ in fundamental ways: one is a crippling chronic mental disease, while the others are temporary, pharmacologically-induced states presently being explored as treatments for mental illnesses. Building towards a deeper understanding of these different alterations of normal consciousness, here we compare the changes in neural dynamics induced by LSD and ketamine (in healthy volunteers) against those associated with schizophrenia, as observed in resting-state M/EEG recordings. While both conditions exhibit increased neural signal diversity, our findings reveal that this is accompanied by an increased transfer entropy from the front to the back of the brain in schizophrenia, versus an overall reduction under the two drugs. Furthermore, we show that these effects can be reproduced via different alterations of standard Bayesian inference applied on a computational model based on the predictive processing framework. In particular, the effects observed under the drugs are modelled as a reduction of the precision of the priors, while the effects of schizophrenia correspond to an increased precision of sensory information. These findings shed new light on the similarities and differences between schizophrenia and two psychotomimetic drug states, and have potential implications for the study of consciousness and future mental health treatments

Effects of External Stimulation on Psychedelic State Neurodynamics

Recent findings have shown that psychedelics reliably enhance brain entropy (understood as neural signal diversity), and this effect has been associated with both acute and long-term psychological outcomes, such as personality changes. These findings are particularly intriguing, given that a decrease of brain entropy is a robust indicator of loss of consciousness (e.g., from wakefulness to sleep). However, little is known about how context impacts the entropy-enhancing effect of psychedelics, which carries important implications for how it can be exploited in, for example, psychedelic psychotherapy. This article investigates how brain entropy is modulated by stimulus manipulation during a psychedelic experience by studying participants under the effects of lysergic acid diethylamide (LSD) or placebo, either with gross state changes (eyes closed vs open) or different stimuli (no stimulus vs music vs video). Results show that while brain entropy increases with LSD under all of the experimental conditions, it exhibits the largest changes when subjects have their eyes closed. Furthermore, brain entropy changes are consistently associated with subjective ratings of the psychedelic experience, but this relationship is disrupted when participants are viewing a video─potentially due to a "competition" between external stimuli and endogenous LSD-induced imagery. Taken together, our findings provide strong quantitative evidence of the role of context in modulating neural dynamics during a psychedelic experience, underlining the importance of performing psychedelic psychotherapy in a suitable environment

Reduced emergent character of neural dynamics in patients with a disrupted connectome

High-level brain functions are widely believed to emerge from the orchestrated activity of multiple neural systems. However, lacking a formal definition and practical quantification of emergence for experimental data, neuroscientists have been unable to empirically test this long-standing conjecture. Here we investigate this fundamental question by leveraging a recently proposed framework known as “Integrated Information Decomposition,” which establishes a principled information-theoretic approach to operationalise and quantify emergence in dynamical systems — including the human brain. By analysing functional MRI data, our results show that the emergent and hierarchical character of neural dynamics is significantly diminished in chronically unresponsive patients suffering from severe brain injury. At a functional level, we demonstrate that emergence capacity is positively correlated with the extent of hierarchical organisation in brain activity. Furthermore, by combining computational approaches from network control theory and whole-brain biophysical modelling, we show that the reduced capacity for emergent and hierarchical dynamics in severely brain-injured patients can be mechanistically explained by disruptions in the patients’ structural connectome. Overall, our results suggest that chronic unresponsiveness resulting from severe brain injury may be related to structural impairment of the fundamental neural infrastructures required for brain dynamics to support emergence

Whole-brain modelling identifies distinct but convergent paths to unconsciousness in anaesthesia and disorders of consciousness

The human brain entertains rich spatiotemporal dynamics, which are drastically reconfigured when consciousness is lost due to anaesthesia or disorders of consciousness (DOC). Here, we sought to identify the neurobiological mechanisms that explain how transient pharmacological intervention and chronic neuroanatomical injury can lead to common reconfigurations of neural activity. We developed and systematically perturbed a neurobiologically realistic model of whole-brain haemodynamic signals. By incorporating PET data about the cortical distribution of GABA receptors, our computational model reveals a key role of spatially-specific local inhibition for reproducing the functional MRI activity observed during anaesthesia with the GABA-ergic agent propofol. Additionally, incorporating diffusion MRI data obtained from DOC patients reveals that the dynamics that characterise loss of consciousness can also emerge from randomised neuroanatomical connectivity. Our results generalise between anaesthesia and DOC datasets, demonstrating how increased inhibition and connectome perturbation represent distinct neurobiological paths towards the characteristic activity of the unconscious brain