Deep Boltzmann Machines as Hierarchical Generative Models of Perceptual Inference in the Cortex

The mammalian neocortex is integral to all aspects of cognition, in particular perception across all sensory modalities. Whether computational principles can be identified that would explain why the cortex is so versatile and capable of adapting to various inputs is not clear. One well-known hypothesis is that the cortex implements a generative model, actively synthesising internal explanations of the sensory input. This ‘analysis by synthesis’ could be instantiated in the top-down connections in the hierarchy of cortical regions, and allow the cortex to evaluate its internal model and thus learn good representations of sensory input over time. Few computational models however exist that implement these principles. In this thesis, we investigate the deep Boltzmann machine (DBM) as a model of analysis by synthesis in the cortex, and demonstrate how three distinct perceptual phenomena can be interpreted in this light: visual hallucinations, bistable perception, and object-based attention. A common thread is that in all cases, the internally synthesised explanations go beyond, or deviate from, what is in the visual input. The DBM was recently introduced in machine learning, but combines several properties of interest for biological application. It constitutes a hierarchical generative model and carries both the semantics of a connectionist neural network and a probabilistic model. Thus, we can consider neuronal mechanisms but also (approximate) probabilistic inference, which has been proposed to underlie cortical processing, and contribute to the ongoing discussion concerning probabilistic or Bayesian models of cognition. Concretely, making use of the model’s capability to synthesise internal representations of sensory input, we model complex visual hallucinations resulting from loss of vision in Charles Bonnet syndrome.We demonstrate that homeostatic regulation of neuronal firing could be the underlying cause, reproduce various aspects of the syndrome, and examine a role for the neuromodulator acetylcholine. Next, we relate bistable perception to approximate, sampling-based probabilistic inference, and show how neuronal adaptation can be incorporated by providing a biological interpretation for a recently developed sampling algorithm. Finally, we explore how analysis by synthesis could be related to attentional feedback processing, employing the generative aspect of the DBM to implement a form of object-based attention. We thus present a model that uniquely combines several computational principles (sampling, neural processing, unsupervised learning) and is general enough to uniquely address a range of distinct perceptual phenomena. The connection to machine learning ensures theoretical grounding and practical evaluation of the underlying principles. Our results lend further credence to the hypothesis of a generative model in the brain, and promise fruitful interaction between neuroscience and Deep Learning approaches

A predictive coding account of bistable perception - a model-based fMRI study

In bistable vision, subjective perception wavers between two interpretations of a constant ambiguous stimulus. This dissociation between conscious perception and sensory stimulation has motivated various empirical studies on the neural correlates of bistable perception, but the neurocomputational mechanism behind endogenous perceptual transitions has remained elusive. Here, we recurred to a generic Bayesian framework of predictive coding and devised a model that casts endogenous perceptual transitions as a consequence of prediction errors emerging from residual evidence for the suppressed percept. Data simulations revealed close similarities between the model’s predictions and key temporal characteristics of perceptual bistability, indicating that the model was able to reproduce bistable perception. Fitting the predictive coding model to behavioural data from an fMRI-experiment on bistable perception, we found a correlation across participants between the model parameter encoding perceptual stabilization and the behaviourally measured frequency of perceptual transitions, corroborating that the model successfully accounted for participants’ perception. Formal model comparison with established models of bistable perception based on mutual inhibition and adaptation, noise or a combination of adaptation and noise was used for the validation of the predictive coding model against the established models. Most importantly, model-based analyses of the fMRI data revealed that prediction error time-courses derived from the predictive coding model correlated with neural signal time-courses in bilateral inferior frontal gyri and anterior insulae. Voxel-wise model selection indicated a superiority of the predictive coding model over conventional analysis approaches in explaining neural activity in these frontal areas, suggesting that frontal cortex encodes prediction errors that mediate endogenous perceptual transitions in bistable perception. Taken together, our current work provides a theoretical framework that allows for the analysis of behavioural and neural data using a predictive coding perspective on bistable perception. In this, our approach posits a crucial role of prediction error signalling for the resolution of perceptual ambiguities

Charles Bonnet Syndrome:Evidence for a Generative Model in the Cortex?

Several theories propose that the cortex implements an internal model to explain, predict, and learn about sensory data, but the nature of this model is unclear. One condition that could be highly informative here is Charles Bonnet syndrome (CBS), where loss of vision leads to complex, vivid visual hallucinations of objects, people, and whole scenes. CBS could be taken as indication that there is a generative model in the brain, specifically one that can synthesise rich, consistent visual representations even in the absence of actual visual input. The processes that lead to CBS are poorly understood. Here, we argue that a model recently introduced in machine learning, the deep Boltzmann machine (DBM), could capture the relevant aspects of (hypothetical) generative processing in the cortex. The DBM carries both the semantics of a probabilistic generative model and of a neural network. The latter allows us to model a concrete neural mechanism that could underlie CBS, namely, homeostatic regulation of neuronal activity. We show that homeostatic plasticity could serve to make the learnt internal model robust against e.g. degradation of sensory input, but overcompensate in the case of CBS, leading to hallucinations. We demonstrate how a wide range of features of CBS can be explained in the model and suggest a potential role for the neuromodulator acetylcholine. This work constitutes the first concrete computational model of CBS and the first application of the DBM as a model in computational neuroscience. Our results lend further credence to the hypothesis of a generative model in the brain

Neural mechanisms of visual awareness and their modulation by social threat

The human brain can extract an enormous wealth of visual information from our surroundings. However, only a fraction of this immense data set ever becomes available to the observer’s awareness. How and why certain information is selected for awareness are questions under active investigation. Following two introductory chapters, this thesis contains six inter-related experimental chapters, through which I will explore two key outstanding questions in this field, using bistable perceptual paradigms to study conscious and non-conscious visual processing in healthy human volunteers. The first major theme focuses on adding new insight into the brain regions and networks that facilitate transfer between non-conscious and conscious modes of visual processing. In Chapters 3 and 4 I will use fMRI, both in task-related and resting-state conditions, to delineate areas, and their interactions (in terms of effective connectivity), which are relevant for transition between different conscious perceptual experiences. In Chapter 5 I will focus on one specific region in the proposed perceptual transition-related network (the frontal eye field) and explore its causal role in access to awareness using repetitive TMS. The second key question explored in this thesis is how social cues in the visual environment influence non-conscious visual processing as well as transition to conscious vision. In Chapter 6 I will study behavioural effects of non-conscious social cues from faces, and the relationship of these effects to focal brain anatomy. Based on finding slower perceptuomotor performance when non-conscious faces contain threatening cues in Chapter 6, I hypothesise that a defensive freezing response is engaged in such situations. The final two experimental chapters will explore the correlates of putative human freezing in the context of non-conscious social threat: using fMRI and psychophysiological measurements to study effects on perceptual transition in Chapter 7, and relating TMS-induced motor-evoked potentials and concurrent psychophysiological measurements to non-conscious perceptuomotor performance in Chapter 8. Taken together, the presented findings shed new light on the network of brain regions involved in transition between non-conscious and conscious modes of visual processing. In addition, they uncover novel mechanisms through which socially relevant visual cues shape our awareness of the visual world, with particular emphasis on the engagement of defensive responses by socially threatening stimuli. The concluding chapter discusses the implications of these findings and explores relevant avenues for future work

Neural mechanisms for reducing uncertainty in 3D depth perception

In order to navigate and interact within their environment, animals must process and interpret sensory information to generate a representation or ‘percept’ of that environment. However, sensory information is invariably noisy, ambiguous, or incomplete due to the constraints of sensory apparatus, and this leads to uncertainty in perceptual interpretation. To overcome these problems, sensory systems have evolved multiple strategies for reducing perceptual uncertainty in the face of uncertain visual input, thus optimizing goal-oriented behaviours. Two available strategies have been observed even in the simplest of neural systems, and are represented in Bayesian formulations of perceptual inference: sensory integration and prior experience. In this thesis, I present a series of studies that examine these processes and the neural mechanisms underlying them in the primate visual system, by studying depth perception in human observers. Chapters 2 & 3 used functional brain imaging to localize cortical areas involved in integrating multiple visual depth cues, which enhance observers’ ability to judge depth. Specifically, we tested which of two possible computational methods the brain uses to combine depth cues. Based on the results we applied disruption techniques to examine whether these select brain regions are critical for depth cue integration. Chapters 4 & 5 addressed the question of how memory systems operating over different time scales interact to resolve perceptual ambiguity when the retinal signal is compatible with more than one 3D interpretation of the world. Finally, we examined the role of higher cortical regions (parietal cortex) in depth perception and the resolution of ambiguous visual input by testing patients with brain lesions

Distributed Bayesian Computation and Self-Organized Learning in Sheets of Spiking Neurons with Local Lateral Inhibition

During the last decade, Bayesian probability theory has emerged as a framework in cognitive science and neuroscience for describing perception, reasoning and learning of mammals. However, our understanding of how probabilistic computations could be organized in the brain, and how the observed connectivity structure of cortical microcircuits supports these calculations, is rudimentary at best. In this study, we investigate statistical inference and self-organized learning in a spatially extended spiking network model, that accommodates both local competitive and large-scale associative aspects of neural information processing, under a unified Bayesian account. Specifically, we show how the spiking dynamics of a recurrent network with lateral excitation and local inhibition in response to distributed spiking input, can be understood as sampling from a variational posterior distribution of a well-defined implicit probabilistic model. This interpretation further permits a rigorous analytical treatment of experience-dependent plasticity on the network level. Using machine learning theory, we derive update rules for neuron and synapse parameters which equate with Hebbian synaptic and homeostatic intrinsic plasticity rules in a neural implementation. In computer simulations, we demonstrate that the interplay of these plasticity rules leads to the emergence of probabilistic local experts that form distributed assemblies of similarly tuned cells communicating through lateral excitatory connections. The resulting sparse distributed spike code of a well-adapted network carries compressed information on salient input features combined with prior experience on correlations among them. Our theory predicts that the emergence of such efficient representations benefits from network architectures in which the range of local inhibition matches the spatial extent of pyramidal cells that share common afferent input

Experimental Manipulation of Action Perception Based on Modeling Computations in Visual Cortex

Action perception, planning and execution is a broad area of study, crucial for future development of clinical therapies treating social cognitive disorders, as well as for building human-computer interaction systems and for giving foundation to an emerging field of developmental robotics. We took interest in basic mechanisms of action perception, and as a model area chose dynamic perception of body motion. The focus of this thesis has been on understanding how perception of actions can be manipulated, how to distill this understanding experimentally, and how to summarize via numerical simulation the neural mechanisms helping explain observed dynamic phenomena. Experimentally we have, first, shown how a careful manipulation of a static object depth cue can in principle modulate perception of actions. We chose the luminance gradient as a model cue, and linked action perception to a perceptual prior previously studied in object recognition – the lighting from above-prior. Second, we have explored the dynamic relationship between representations of actions that are naturally observed in spatiotemporal proximity. We have shown an adaptation aftereffect that may speak of brain mechanisms encoding social interactions. To qualitatively capture neural mechanisms behind ours and previous findings, we have additionally appealed to the perceptual bistability phenomenon. Bistable perception refers to the ability to spontaneously switch between two perceptual alternatives arising from an observation of a single stimulus. Addition of depth cues to biological motion stimulus resolves depth-ambiguity. To account for neural dynamics as well as for modulation of action percept by light source position, we used a combined architecture with a convolutional neural network computing shading and form features in biological motion stimuli, and a 2-dimensional neural field coding for walking direction and body configuration in the gait cycle. This single unified model matches experimentally observed switching statistics, dependence of recognized walking direction on the light source position, and makes a prediction for the adaptation aftereffect in perception of biological motion

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

25th Annual Computational Neuroscience Meeting: CNS-2016

Abstracts of the 25th Annual Computational Neuroscience Meeting: CNS-2016 Seogwipo City, Jeju-do, South Korea. 2–7 July 201