I expect, therefore I see: individual differences in visual awareness

Predictive processing theories posit that awareness of the visual world emerges as the brain engages in predictive inference about the causes of its sensory input. At each level of the processing hierarchy top-down predictions are corrected by bottom-up sensory prediction error to form behaviourally optimal inferences about the state of the visual world. Research suggests there may be individual differences in predictive processing mechanisms such that some individuals are more reliant on prior knowledge, whereas others assign more weight to sensory evidence. Predictive processing biases are thought to manifest in a range of typical and atypical perceptual experiences including proneness to perceptual illusions, sensory sensitivity in autism, and hallucinations in psychosis. The overarching aim of this thesis was to investigate whether in the general population predictive processing biases predict individual differences in visual awareness. Change blindness was selected as the central paradigm of investigation, as it can be conceptualised as a failure to incorporate a novel change into the current prediction about the state of the visual world. The empirical work in Chapter 2 aimed to characterise individual differences in visual change detection using naturalistic scenes and to identify the perceptual and cognitive measures that predict noticing ability. There were reliable individual differences in change detection that generalised to ecologically valid displays. The ability to notice visual changes was predicted by the strength and stability of perceptual predictions, as measured by the accuracy of visual short-term memory and attentional control in the face of distractors. In Chapter 3 I used voxel-based-morphometry to investigate whether inter-individual variability in brain structure predicts individual differences in visual awareness. The latter was assessed by the change blindness task as well as its strongest predictor measures (visual short-term memory, attentional capture, and perceptual rivalry). Regions of interest (ROIs) were selected in the parietal and visual cortices based on previous evidence that these areas are causally involved in the awareness of visual stimuli. This study aimed to discover whether the average grey matter density in the ROIs predict susceptibility to CB. The ROI-based analyses revealed the average grey matter density in left posterior parietal cortex predicted visual short-term memory accuracy but none of the other hypothesised relationships were significant. Chapter 4 aimed to measure individual differences in the reliance on prior knowledge by employing the Mooney face detection task. In this task participants disambiguated faces in two-tone degraded images before and after the presentation of the original versions of the images. Better change detection was predicted by Mooney face detection without any prior knowledge of the images, a measure of ‘perceptual closure’ or an ability to generate a gestalt of a scene. The attention to detail subscale of the autism spectrum also predicted superior change detection. Reliance on prior knowledge in visual perception (assessed by improvement in Mooney face detection after seeing original images) did not consistently predict atypical perceptual experiences associated with the autism spectrum or schizotypy. Chapter 5 was an investigation into, firstly, whether there is a general predictive processing bias, which manifests across different methods of inducing prior knowledge, or whether such a bias is paradigm-specific and, secondly, whether reliance on priors predicts perceptual experiences and traits. All prior manipulations in this study lead to an increased tendency to see the expected stimulus in a binocular rivalry display, except adaptation, which lead to a suppression of visual awareness. Attentional control, perceptual priming, expectancy, and imagery loaded onto a common factor, suggesting that the strength of selective attention is closely linked with the facilitatory effect of expectation. The strength of adaptation predicted superior change detection and perceptual priming predicted the propensity to experience perceptual illusions. Taken together, these findings suggest that there are reliable individual differences in visual change detection, and these are predicted by the strength of visual short-term memory representations, attentional control, perceptual closure ability, as well as the strength of low-level adaptation. Possessing expectations facilitates the entry of the corresponding percept into awareness, irrespective of the method of prior induction. The facilitatory effect that priors exert on visual awareness across different methods is closely linked with the ability to exert attentional control. This suggests that the effects of expectations on awareness may be attentional. However, predictive processing biases were method-specific in that a facilitatory effect using one prior induction method will not necessarily predict the magnitude of the effect using a different method. Some prior effects (e.g., perceptual priming, imagery, and adaptation) yielded correlations with perceptual experiences and traits in the general population. As the research in this thesis is correlational, future studies will need to delineate the effects of expectation, attention, and adaptation on visual awareness and explore the neural representations of these mechanisms

A perceptual comparison of empirical and predictive region-of-interest video

When viewing multimedia presentations, a user only attends to a relatively small part of the video display at any one point in time. By shifting allocation of bandwidth from peripheral areas to those locations where a user’s gaze is more likely to rest, attentive displays can be produced. Attentive displays aim to reduce resource requirements while minimizing negative user perception—understood in this paper as not only a user’s ability to assimilate and understand information but also his/her subjective satisfaction with the video content. This paper introduces and discusses a perceptual comparison between two region-of-interest display (RoID) adaptation techniques. A RoID is an attentive display where bandwidth has been preallocated around measured or highly probable areas of user gaze. In this paper, video content was manipulated using two sources of data: empirical measured data (captured using eye-tracking technology) and predictive data (calculated from the physical characteristics of the video data). Results show that display adaptation causes significant variation in users’ understanding of specific multimedia content. Interestingly, RoID adaptation and the type of video being presented both affect user perception of video quality. Moreover, the use of frame rates less than 15 frames per second, for any video adaptation technique, caused a significant reduction in user perceived quality, suggesting that although users are aware of video quality reduction, it does impact level of information assimilation and understanding. Results also highlight that user level of enjoyment is significantly affected by the type of video yet is not as affected by the quality or type of video adaptation—an interesting implication in the field of entertainment

Top-down effects on early visual processing in humans: a predictive coding framework

An increasing number of human electroencephalography (EEG) studies examining the earliest component of the visual evoked potential, the so-called C1, have cast doubts on the previously prevalent notion that this component is impermeable to top-down effects. This article reviews the original studies that (i) described the C1, (ii) linked it to primary visual cortex (V1) activity, and (iii) suggested that its electrophysiological characteristics are exclusively determined by low-level stimulus attributes, particularly the spatial position of the stimulus within the visual field. We then describe conflicting evidence from animal studies and human neuroimaging experiments and provide an overview of recent EEG and magnetoencephalography (MEG) work showing that initial V1 activity in humans may be strongly modulated by higher-level cognitive factors. Finally, we formulate a theoretical framework for understanding top-down effects on early visual processing in terms of predictive coding

Can hierarchical predictive coding explain binocular rivalry?

Hohwy et al.’s (2008) model of binocular rivalry (BR) is taken as a classic illustration of predictive coding’s explanatory power. I revisit the account and show that it cannot explain the role of reward in BR. I then consider a more recent version of Bayesian model averaging, which recasts the role of reward in (BR) in terms of optimism bias. If we accept this account, however, then we must reconsider our conception of perception. On this latter view, I argue, organisms engage in what amounts to policy-driven, motivated perception

Temporal characteristics of the influence of punishment on perceptual decision making in the human brain

Perceptual decision making is the process by which information from sensory systems is combined and used to influence our behavior. In addition to the sensory input, this process can be affected by other factors, such as reward and punishment for correct and incorrect responses. To investigate the temporal dynamics of how monetary punishment influences perceptual decision making in humans, we collected electroencephalography (EEG) data during a perceptual categorization task whereby the punishment level for incorrect responses was parametrically manipulated across blocks of trials. Behaviorally, we observed improved accuracy for high relative to low punishment levels. Using multivariate linear discriminant analysis of the EEG, we identified multiple punishment-induced discriminating components with spatially distinct scalp topographies. Compared with components related to sensory evidence, components discriminating punishment levels appeared later in the trial, suggesting that punishment affects primarily late postsensory, decision-related processing. Crucially, the amplitude of these punishment components across participants was predictive of the size of the behavioral improvements induced by punishment. Finally, trial-by-trial changes in prestimulus oscillatory activity in the alpha and gamma bands were good predictors of the amplitude of these components. We discuss these findings in the context of increased motivation/attention, resulting from increases in punishment, which in turn yields improved decision-related processing

Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future

Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)

The Problem of Mental Action

In mental action there is no motor output to be controlled and no sensory input vector that could be manipulated by bodily movement. It is therefore unclear whether this specific target phenomenon can be accommodated under the predictive processing framework at all, or if the concept of “active inference” can be adapted to this highly relevant explanatory domain. This contribution puts the phenomenon of mental action into explicit focus by introducing a set of novel conceptual instruments and developing a first positive model, concentrating on epistemic mental actions and epistemic self-control. Action initiation is a functionally adequate form of self-deception; mental actions are a specific form of predictive control of effective connectivity, accompanied and possibly even functionally mediated by a conscious “epistemic agent model”. The overall process is aimed at increasing the epistemic value of pre-existing states in the conscious self-model, without causally looping through sensory sheets or using the non-neural body as an instrument for active inference

Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene with eye movements

How does the brain maintain stable fusion of 3D scenes when the eyes move? Every eye movement causes each retinal position to process a different set of scenic features, and thus the brain needs to binocularly fuse new combinations of features at each position after an eye movement. Despite these breaks in retinotopic fusion due to each movement, previously fused representations of a scene in depth often appear stable. The 3D ARTSCAN neural model proposes how the brain does this by unifying concepts about how multiple cortical areas in the What and Where cortical streams interact to coordinate processes of 3D boundary and surface perception, spatial attention, invariant object category learning, predictive remapping, eye movement control, and learned coordinate transformations. The model explains data from single neuron and psychophysical studies of covert visual attention shifts prior to eye movements. The model further clarifies how perceptual, attentional, and cognitive interactions among multiple brain regions (LGN, V1, V2, V3A, V4, MT, MST, PPC, LIP, ITp, ITa, SC) may accomplish predictive remapping as part of the process whereby view-invariant object categories are learned. These results build upon earlier neural models of 3D vision and figure-ground separation and the learning of invariant object categories as the eyes freely scan a scene. A key process concerns how an object's surface representation generates a form-fitting distribution of spatial attention, or attentional shroud, in parietal cortex that helps maintain the stability of multiple perceptual and cognitive processes. Predictive eye movement signals maintain the stability of the shroud, as well as of binocularly fused perceptual boundaries and surface representations.Published versio

How active perception and attractor dynamics shape perceptual categorization: A computational model

We propose a computational model of perceptual categorization that fuses elements of grounded and sensorimotor theories of cognition with dynamic models of decision-making. We assume that category information consists in anticipated patterns of agent–environment interactions that can be elicited through overt or covert (simulated) eye movements, object manipulation, etc. This information is firstly encoded when category information is acquired, and then re-enacted during perceptual categorization. The perceptual categorization consists in a dynamic competition between attractors that encode the sensorimotor patterns typical of each category; action prediction success counts as ‘‘evidence’’ for a given category and contributes to falling into the corresponding attractor. The evidence accumulation process is guided by an active perception loop, and the active exploration of objects (e.g., visual exploration) aims at eliciting expected sensorimotor patterns that count as evidence for the object category. We present a computational model incorporating these elements and describing action prediction, active perception, and attractor dynamics as key elements of perceptual categorizations. We test the model in three simulated perceptual categorization tasks, and we discuss its relevance for grounded and sensorimotor theories of cognition.Peer reviewe