The role of predictive processing in conscious access and regularity learning across sensory domains

To increase fitness for survival, organisms not only passively react to environmental changes but also actively predict future events to prepare for potential hazards within their environment. Accumulating evidence indicates that the human brain is a remarkable predictive machine which constantly models causal relationships and predicts future events. This ‘predictive processing’ framework, a prediction-based form of Bayesian inference, states that the brain continuously generates and updates predictions about incoming sensory signals. This framework has been showing notable explanatory power in understanding the mechanisms behind both human behaviour and neurophysiological data and elegantly specifies the underlying computational principles of the neural system. However, even though predictive processing has the potential to provide a unified theory of the brain (Karl Friston, 2010), we still have a limited understanding about fundamental aspects of this model, such as how it deals with different types of information, learns statistical regularities and perhaps most fundamentally of all what its relationship to conscious experience is. This thesis aims to investigate the major gaps in our current understanding of the predictive processing framework via a series of studies. Study 1 investigated the fundamental relationship between unconscious statistical inference reflected by predictive processing and conscious access. It demonstrated that predictions that are in line with sensory evidence accelerate conscious access. Study 2 investigated how low level information within the sensory hierarchy is dealt with by predictive processing and regularity learning mechanisms through “perceptual echo” in which the cross-correlation between a sequence of randomly fluctuating luminance values and occipital electrophysiological (EEG) signals exhibits a long-lasting periodic (~100ms cycle) reverberation of the input stimulus. This study identified a new form of regularity learning and the results demonstrate that the perceptual echo may reflect an iterative learning process, governed by predictive processing. Study 3 investigated how supra-modal predictive processing is capable of learning regularities of temporal duration and also temporal predictions about future events. This study revealed a supramodal temporal prediction mechanism which processes auditory and visual temporal information and integrates information from the duration and rhythmic structures of events. Together these studies provide a global picture of predictive processing and regularity learning across differing types of predictive information

Vividness, Consciousness, and Mental Imagery

Today in many studies, mental images are still either treated as conscious by definition, or as empirical operations implicit to completing some type of task, such as the measurement of reaction time in mental rotation, an underlying mental image is assumed, but there is no direct determination of whether it is conscious or not. The vividness of mental images is a potentially helpful construct which may be suitable, as it may correspond to consciousness or aspects of the consciousness of images. In this context, a complicating factor seems to be the surprising variety in what is meant by the term vividness or how it is used or theorized. To fill some of the gaps, the goal of the present Special Issue is to create a publication outlet where authors can fully explore through sound research the missing theoretical and empirical links between vividness, consciousness and mental imagery across disciplines, neuroscience, psychology, philosophy, cognitive science, to mention the most obvious ones, as well as transdisciplinary methodological (single, combined, or multiple) approaches

ROLE OF INHIBITION AND SPIKING VARIABILITY IN ORTHO- AND RETRONASAL OLFACTORY PROCESSING

Odor perception is the impetus for important animal behaviors, most pertinently for feeding, but also for mating and communication. There are two predominate modes of odor processing: odors pass through the front of nose (ortho) while inhaling and sniffing, or through the rear (retro) during exhalation and while eating and drinking. Despite the importance of olfaction for an animal’s well-being and specifically that ortho and retro naturally occur, it is unknown whether the modality (ortho versus retro) is transmitted to cortical brain regions, which could significantly instruct how odors are processed. Prior imaging studies show different brain activity for the two modes, even with identical odors. However, odors are first processed via coordinated spiking of neurons in the olfactory bulb (OB) before being relayed downstream to higher cortical regions. Thus, we investigate responses of mitral cells (MC), one of principle neurons in OB, to ortho and retro stimulus to elucidate how the OB processes and codes this information. We analyze our collected in vivo rat data to inform modeling of the OB circuitry and MC responses to both modes of olfaction. Our efforts show that the OB does indeed process odors differently and that the temporal profile of each stimulus route to the OB is crucial for distinguishing ortho and retro odors. Additionally, we detail the rich spiking dynamics observed in our MC model and use a phenomenological model to explain the unexpected non-monotonic spike variability observed as weak-to-moderate background noise increases. Lastly in both anesthetized and awake rodents, we show that MCs with synaptic connections to cortical regions reliably transmit ortho versus retro input stimulus information. Drug manipulation affecting GABAA-mediated synaptic inhibition leads to changes in decoding of ortho/retro and only affects firing response for one of the two modes. We have not only shown that ortho versus retro information is encoded to downstream brain regions, but with models and analysis, we uncover the network dynamics that promote this encoding

The temporal pattern of impulses in primary afferents analogously encodes touch and hearing information

An open question in neuroscience is the contribution of temporal relations between individual impulses in primary afferents in conveying sensory information. We investigated this question in touch and hearing, while looking for any shared coding scheme. In both systems, we artificially induced temporally diverse afferent impulse trains and probed the evoked perceptions in human subjects using psychophysical techniques. First, we investigated whether the temporal structure of a fixed number of impulses conveys information about the magnitude of tactile intensity. We found that clustering the impulses into periodic bursts elicited graded increases of intensity as a function of burst impulse count, even though fewer afferents were recruited throughout the longer bursts. The interval between successive bursts of peripheral neural activity (the burst-gap) has been demonstrated in our lab to be the most prominent temporal feature for coding skin vibration frequency, as opposed to either spike rate or periodicity. Given the similarities between tactile and auditory systems, second, we explored the auditory system for an equivalent neural coding strategy. By using brief acoustic pulses, we showed that the burst-gap is a shared temporal code for pitch perception between the modalities. Following this evidence of parallels in temporal frequency processing, we next assessed the perceptual frequency equivalence between the two modalities using auditory and tactile pulse stimuli of simple and complex temporal features in cross-sensory frequency discrimination experiments. Identical temporal stimulation patterns in tactile and auditory afferents produced equivalent perceived frequencies, suggesting an analogous temporal frequency computation mechanism. The new insights into encoding tactile intensity through clustering of fixed charge electric pulses into bursts suggest a novel approach to convey varying contact forces to neural interface users, requiring no modulation of either stimulation current or base pulse frequency. Increasing control of the temporal patterning of pulses in cochlear implant users might improve pitch perception and speech comprehension. The perceptual correspondence between touch and hearing not only suggests the possibility of establishing cross-modal comparison standards for robust psychophysical investigations, but also supports the plausibility of cross-sensory substitution devices

Visual processing in the human brain: Investigating deviance detection from a predictive coding perspective

According to predictive coding, the brain gives extra processing to unpredicted events that disrupt anticipated patterns. To adapt to these events, the brain continually extracts statistical regularities about sensory input from past input. When something unpredicted occurs, it produces an error. In vision, this can be shown by the visual mismatch negativity (vMMN) in event-related potentials (ERPs). The vMMN reaches its maximum amplitude between 150 and 300 ms after the onset of an irregular, deviant event in a sequence of otherwise regular, standard events and it is usually measured from areas on the scalp closest to the visual cortices (e.g., parieto-occipital areas). Attention toward a deviant is not necessary to generate the vMMN, suggesting that regularities and irregularities are pre-attentively encoded and detected, respectively. Although vMMN research continues to grow, there are still unanswered questions about it. This thesis focuses on clarifying some of these issues, asking whether the type or size of the difference between predicted and unpredicted visual input (i.e., the magnitude of deviance) or visual field in which deviance occurs can affect the vMMN. To remedy this, I manipulated these facets across four studies. My thesis was that local aspects of change detection, such as the magnitude of deviance, affect the brain’s error response to unpredicted input, evidenced by the vMMN. A conclusion regarding the effect of magnitude of deviance, the type of change, or visual field on the vMMN was not possible given that (1) ERPs to rule-based deviants and standards did not differ where participants found it difficult to detect irregularities in visual input, and (2) changes in basic properties of well-controlled visual stimuli do not evoke the vMMN. Subsequently, my thesis became that isolated changes in basic properties of visual input do not evoke the vMMN, perhaps because these changes are detected and resolved prior to the vMMN. Instead, this thesis provides evidence for an earlier deviant-related positivity for changes in low-level features of visual input. This is the first report of a possible pre-vMMN positive prediction error and represents a significant and original contribution to the wider field

Contributions of Familiarity and Chunking to Visual Working Memory Capacity

Visual working memory (VWM) is responsible for the temporary storage of visual information required for perception and cognition. The capacity of VWM is surprisingly limited to three or four items. Despite decades of research, the nature of the capacity limit is still unclear, in part due to uncertainty about the main factors contributing to this limit. We approached this issue by exploring two instances in which memory performance is enhanced. Firstly, while controlling stimulus complexity and similarity, familiarity produced significant increases in both encoding rate and capacity. However, familiarity gained from training observers to simply recognise the stimuli did not produce any benefits for change detection. Secondly, the inclusion of statistical regularities in the displays produced significantly improved recall. However, only subjects with explicit awareness of the statistical regularities showed improvement, whereas unaware subjects showed no change in their recall performance. We extended this result by observing whether contralateral delay activity (CDA), a neural marker of the number of item-based representations held in VWM, reduces with explicit chunking. Although recall performance was significantly better, the CDA did not appear to index equivalent number of chunks, suggesting that online representations do not change with the use of explicit chunking. Instead, the behavioural benefit appears to rely on retrieval of a long-term memory representation (LTM) when recall is tested. These results indicate a major influence of LTM in guiding VWM performance. Behavioural data collected at the end of the trial, such as change detection or probed recall, appear inadequate for fully examining the nature of VWM. An embedded-process framework, in which activated LTM representations can fluidly shift into the focus of attention, is useful in interpreting these results and understanding the cognitive processes involved in memory

Music adapting to the brain: From diffusion chains to neurophysiology

During the last decade, the use of experimental approaches on cultural evolution research has provided novel insights, and supported theoretical predictions, on the principles driving the evolution of human cultural systems. Laboratory simulations of language evolution showed how general-domain constraints on learning, in addition to pressures for language to be expressive, may be responsible for the emergence of linguistic structure. Languages change when culturally transmitted, adapting to fit, among all, the cognitive abilities of their users. As a result, they become regular and compressed, easier to acquire and reproduce. Although a similar theory has been recently extended to the musical domain, the empirical investigation in this field is still scarce. In addition, no study to our knowledge directly addressed the role of cognitive constraints in cultural transmission with neurophysiological investigation. In my thesis I addressed both these issues with a combination of behavioral and neurophysiological methods, in three experimental studies. In study 1 (Chapter 2), I examined the evolution of structural regularities in artificial melodic systems while they were being transmitted across individuals via coordination and alignment. To this purpose I used a new laboratory model of music transmission: the multi-generational signaling games (MGSGs), a variant of the signaling games. This model combines classical aspects of lab-based semiotic models of communication, coordination and interaction (horizontal transmission), with the vertical transmission across generations of the iterated learning model (vertical transmission). Here, two-person signaling games are organized in diffusion chains of several individuals (generations). In each game, the two players (a sender and a receiver) must agree on a common code - here a miniature system where melodic riffs refer to emotions. The receiver in one game becomes the sender in the next game, possibly retransmitting the code previously learned to another generation of participants, and so on to complete the diffusion chain. I observed the gradual evolution of several structures features of musical phrases over generations: proximity, continuity, symmetry, and melodic compression. Crucially, these features are found in most of musical cultures of the world. I argue that we tapped into universal processing mechanisms of structured sequence processing, possibly at work in the evolution of real music. In study 2 (Chapter 3), I explored the link between cultural adaptation and neural information processing. To this purpose, I combined behavioral and EEG study on 2 successive days. I show that the latency of the mismatch negativity (MMN) recorded in a pre-attentive auditory sequence processing task on day 1, predicts how well participants learn and transmit an artificial tone system with affective semantics in two signaling games on day 2. Notably, MMN latencies also predict which structural changes are introduced by participants into the artificial tone system. In study 3 (Chapter 4), I replicated and extended behavioral and neurophysiological findings on the temporal domain of music, with two independent experiments. In the first experiment, I used MGSGs as a laboratory model of cultural evolution of rhythmic equitone patterns referring to distinct emotions. As a result of transmission, rhythms developed a universal property of music structure, namely temporal regularity (or isochronicity). In the second experiment, I anchored this result with neural predictors. I showed that neural information processing capabilities of individuals, as measured with the MMN on day 1, can predict learning, transmission, and regularization of rhythmic patterns in signaling games on day 2. In agreement with study 2, I observe that MMN brain timing may reflect the efficiency of sensory systems to process auditory patterns. Functional differences in those systems, across individuals, may produce a different sensitivity to pressures for regularities in the cultural system. Finally, I argue that neural variability can be an important source of variability of cultural traits in a population. My work is the first to systematically describe the emergence of structural properties of melodic and rhythmic systems in the laboratory, using an explicit game-theoretic model of cultural transmission in which agents freely interact and exchange information. Critically, it provides the first demonstration that social learning, transmission, and cultural adaptation are constrained and driven by individual differences in the functional organization of sensory systems