Top-Down Processing: Top-Down Network Combines Back-Propagation with Attention

Early neural network models relied exclusively on bottom-up processing going from the input signals to higher-level representations. Many recent models also incorporate top-down networks going in the opposite direction. Top-down processing in deep learning models plays two primary roles: learning and directing attention. These two roles are accomplished in current models through distinct mechanisms. While top-down attention is often implemented by extending the model's architecture with additional units that propagate information from high to low levels of the network, learning is typically accomplished by an external learning algorithm such as back-propagation. In the current work, we present an integration of the two functions above, which appear unrelated, using a single unified mechanism. We propose a novel symmetric bottom-up top-down network structure that can integrate standard bottom-up networks with a symmetric top-down counterpart, allowing each network to guide and influence the other. The same top-down network is being used for both learning, via back-propagating feedback signals, and at the same time also for top-down attention, by guiding the bottom-up network to perform a selected task. We show that our method achieves competitive performance on a standard multi-task learning benchmark. Yet, we rely on standard single-task architectures and optimizers, without any task-specific parameters. Additionally, our learning algorithm addresses in a new way some neuroscience issues that arise in biological modeling of learning in the brain

Word contexts enhance the neural representation of individual letters in early visual cortex

Visual context facilitates perception, but how this is neurally implemented remains unclear. One example of contextual facilitation is found in reading, where letters are more easily identified when embedded in a word. Bottom-up models explain this word advantage as a post-perceptual decision bias, while top-down models propose that word contexts enhance perception itself. Here, we arbitrate between these accounts by presenting words and nonwords and probing the representational fidelity of individual letters using functional magnetic resonance imaging. In line with top-down models, we find that word contexts enhance letter representations in early visual cortex. Moreover, we observe increased coupling between letter information in visual cortex and brain activity in key areas of the reading network, suggesting these areas may be the source of the enhancement. Our results provide evidence for top-down representational enhancement in word recognition, demonstrating that word contexts can modulate perceptual processing already at the earliest visual regions

Preferential effect of isoflurane on top-down vs. bottom-up pathways in sensory cortex

The mechanism of loss of consciousness (LOC) under anesthesia is unknown. Because consciousness depends on activity in the cortico-thalamic network, anesthetic actions on this network are likely critical for LOC. Competing theories stress the importance of anesthetic actions on bottom-up “core” thalamo-cortical (TC) vs. top-down cortico-cortical (CC) and matrix TC connections. We tested these models using laminar recordings in rat auditory cortex in vivo and murine brain slices. We selectively activated bottom-up vs. top-down afferent pathways using sensory stimuli in vivo and electrical stimulation in brain slices, and compared effects of isoflurane on responses evoked via the two pathways. Auditory stimuli in vivo and core TC afferent stimulation in brain slices evoked short latency current sinks in middle layers, consistent with activation of core TC afferents. By contrast, visual stimuli in vivo and stimulation of CC and matrix TC afferents in brain slices evoked responses mainly in superficial and deep layers, consistent with projection patterns of top-down afferents that carry visual information to auditory cortex. Responses to auditory stimuli in vivo and core TC afferents in brain slices were significantly less affected by isoflurane compared to responses triggered by visual stimuli in vivo and CC/matrix TC afferents in slices. At a just-hypnotic dose in vivo, auditory responses were enhanced by isoflurane, whereas visual responses were dramatically reduced. At a comparable concentration in slices, isoflurane suppressed both core TC and CC/matrix TC responses, but the effect on the latter responses was far greater than on core TC responses, indicating that at least part of the differential effects observed in vivo were due to local actions of isoflurane in auditory cortex. These data support a model in which disruption of top-down connectivity contributes to anesthesia-induced LOC, and have implications for understanding the neural basis of consciousness

Whole-Brain Models to Explore Altered States of Consciousness from the Bottom Up.

The scope of human consciousness includes states departing from what most of us experience as ordinary wakefulness. These altered states of consciousness constitute a prime opportunity to study how global changes in brain activity relate to different varieties of subjective experience. We consider the problem of explaining how global signatures of altered consciousness arise from the interplay between large-scale connectivity and local dynamical rules that can be traced to known properties of neural tissue. For this purpose, we advocate a research program aimed at bridging the gap between bottom-up generative models of whole-brain activity and the top-down signatures proposed by theories of consciousness. Throughout this paper, we define altered states of consciousness, discuss relevant signatures of consciousness observed in brain activity, and introduce whole-brain models to explore the biophysics of altered consciousness from the bottom-up. We discuss the potential of our proposal in view of the current state of the art, give specific examples of how this research agenda might play out, and emphasize how a systematic investigation of altered states of consciousness via bottom-up modeling may help us better understand the biophysical, informational, and dynamical underpinnings of consciousness

Self-regulation learning as active inference: dynamic causal modeling of an fMRI neurofeedback task

Introduction: Learning to self-regulate brain activity by neurofeedback has been shown to lead to changes in the brain and behavior, with beneficial clinical and non-clinical outcomes. Neurofeedback uses a brain-computer interface to guide participants to change some feature of their brain activity. However, the neural mechanism of self-regulation learning remains unclear, with only 50% of the participants succeeding in achieving it. To bridge this knowledge gap, our study delves into the neural mechanisms of self-regulation learning via neurofeedback and investigates the brain processes associated with successful brain self-regulation. Methods: We study the neural underpinnings of self-regulation learning by employing dynamical causal modeling (DCM) in conjunction with real-time functional MRI data. The study involved a cohort of 18 participants undergoing neurofeedback training targeting the supplementary motor area. A critical focus was the comparison between top-down hierarchical connectivity models proposed by Active Inference and alternative bottom-up connectivity models like reinforcement learning. Results: Our analysis revealed a crucial distinction in brain connectivity patterns between successful and non-successful learners. Particularly, successful learners evinced a significantly stronger top-down effective connectivity towards the target area implicated in self-regulation. This heightened top-down network engagement closely resembles the patterns observed in goal-oriented and cognitive control studies, shedding light on the intricate cognitive processes intertwined with self-regulation learning. Discussion: The findings from our investigation underscore the significance of cognitive mechanisms in the process of self-regulation learning through neurofeedback. The observed stronger top-down effective connectivity in successful learners indicates the involvement of hierarchical cognitive control, which aligns with the tenets of Active Inference. This study contributes to a deeper understanding of the neural dynamics behind successful self-regulation learning and provides insights into the potential cognitive architecture underpinning this process

Brain Learning, Attention, and Consciousness

The processes whereby our brains continue to learn about a changing world in a stable fashion throughout life are proposed to lead to conscious experiences. These processes include the learning of top-down expectations, the matching of these expectations against bottom-up data, the focusing of attention upon the expected clusters of information, and the development of resonant states between bottom-up and top-down processes as they reach an attentive consensus between what is expected and what is there in the outside world. It is suggested that all conscious states in the brain are resonant states, and that these resonant states trigger learning of sensory and cognitive representations. The model which summarize these concepts are therefore called Adaptive Resonance Theory, or ART, models. Psychophysical and neurobiological data in support of ART are presented from early vision, visual object recognition, auditory streaming, variable-rate speech perception, somatosensory perception, and cognitive-emotional interactions, among others. It is noted that ART mechanisms seem to be operative at all levels of the visual system, and it is proposed how these mechanisms are realized by known laminar circuits of visual cortex. It is predicted that the same circuit realization of ART mechanisms will be found in the laminar circuits of all sensory and cognitive neocortex. Concepts and data are summarized concerning how some visual percepts may be visibly, or modally, perceived, whereas amoral percepts may be consciously recognized even though they are perceptually invisible. It is also suggested that sensory and cognitive processing in the What processing stream of the brain obey top-down matching and learning laws that arc often complementary to those used for spatial and motor processing in the brain's Where processing stream. This enables our sensory and cognitive representations to maintain their stability a.s we learn more about the world, while allowing spatial and motor representations to forget learned maps and gains that are no longer appropriate as our bodies develop and grow from infanthood to adulthood. Procedural memories are proposed to be unconscious because the inhibitory matching process that supports these spatial and motor processes cannot lead to resonance.Defense Advance Research Projects Agency; Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657); National Science Foundation (IRI-97-20333

Mindfulness training applied to addiction therapy: insights into the neural mechanisms of positive behavioral change

Dual-process models from neuroscience suggest that addiction is driven by dysregulated interactions between bottom-up neural processes underpinning reward learning and top-down neural functions subserving executive function. Over time, drug use causes atrophy in prefrontally mediated cognitive control networks and hijacks striatal circuits devoted to processing natural rewards in service of compulsive seeking of drug-related reward. In essence, mindfulness-based interventions (MBIs) can be conceptualized as mental training programs for exercising, strengthening, and remediating these functional brain networks. This review describes how MBIs may remediate addiction by regulating frontostriatal circuits, thereby restoring an adaptive balance between these top-down and bottom-up processes. Empirical evidence is presented suggesting that MBIs facilitate cognitive control over drug-related automaticity, attentional bias, and drug cue reactivity, while enhancing responsiveness to natural rewards. Findings from the literature are incorporated into an integrative account of the neural mechanisms of mindfulness-based therapies for effecting positive behavior change in the context of addiction recovery. Implications of our theoretical framework are presented with respect to how these insights can inform the addiction therapy process.National Institute of Health (NIH) award to ELG (R34DA037005

Processing Narrative Coherence: Towards a top-down model of discourse.

Models of discourse and narration elaborated within the classical compositional framework have been characterized as bottom-up models, according to which discourse analysis proceeds incrementally, from phrase and sentence local meaning to discourse global meaning. In this paper we will argue against these models. Assuming as a case study the issue of discourse coherence, we suggest that the assessment of coherence is a top-down process, in which the construction of a situational interpretation at the global meaning level guides local meaning analysis. In support of our hypothesis, we explore the role of executive functions (brain functions involved in planning and organization of goal-oriented behaviors) in coherence’s establishment, discussing the results of several studies on narrative abilities of patients with brain injuries. We suggest that, compared to other models of discourse processing focused on comprehension, our model is a viable candidate for an integrated account of discourse comprehension and production