Dynamic competition between large-scale functional networks differentiates fear conditioning and extinction in humans.

The high evolutionary value of learning when to respond to threats or when to inhibit previously learned associations after changing threat contingencies is reflected in dedicated networks in the animal and human brain. Recent evidence further suggests that adaptive learning may be dependent on the dynamic interaction of meta-stable functional brain networks. However, it is still unclear which functional brain networks compete with each other to facilitate associative learning and how changes in threat contingencies affect this competition. The aim of this study was to assess the dynamic competition between large-scale networks related to associative learning in the human brain by combining a repeated differential conditioning and extinction paradigm with independent component analysis of functional magnetic resonance imaging data. The results (i) identify three task-related networks involved in initial and sustained conditioning as well as extinction, and demonstrate that (ii) the two main networks that underlie sustained conditioning and extinction are anti-correlated with each other and (iii) the dynamic competition between these two networks is modulated in response to changes in associative contingencies. These findings provide novel evidence for the view that dynamic competition between large-scale functional networks differentiates fear conditioning from extinction learning in the healthy brain and suggest that dysfunctional network dynamics might contribute to learning-related neuropsychiatric disorders

The Multi-Dimensional Contributions of Prefrontal Circuits to Emotion Regulation during Adulthood and Critical Stages of Development

The prefrontal cortex (PFC) plays a pivotal role in regulating our emotions. The importance of ventromedial regions in emotion regulation, including the ventral sector of the medial PFC, the medial sector of the orbital cortex and subgenual cingulate cortex, have been recognized for a long time. However, it is increasingly apparent that lateral and dorsal regions of the PFC, as well as neighbouring dorsal anterior cingulate cortex, also play a role. Defining the underlying psychological mechanisms by which these functionally distinct regions modulate emotions and the nature and extent of their interactions is a critical step towards better stratification of the symptoms of mood and anxiety disorders. It is also important to extend our understanding of these prefrontal circuits in development. Specifically, it is important to determine whether they exhibit differential sensitivity to perturbations by known risk factors such as stress and inflammation at distinct developmental epochs. This Special Issue brings together the most recent research in humans and other animals that addresses these important issues, and in doing so, highlights the value of the translational approach

A neural network model of normal and abnormal learning and memory consolidation

The amygdala and hippocampus interact with thalamocortical systems to regulate cognitive-emotional learning, and lesions of amygdala, hippocampus, thalamus, and cortex have different effects depending on the phase of learning when they occur. In examining eyeblink conditioning data, several questions arise: Why is the hippocampus needed for trace conditioning where there is a temporal gap between the conditioned stimulus offset and the onset of the unconditioned stimulus, but not needed for delay conditioning where stimuli temporally overlap and co-terminate? Why do amygdala lesions made before or immediately after training decelerate conditioning while those made later have no impact on conditioned behavior? Why do thalamic lesions degrade trace conditioning more than delay conditioning? Why do hippocampal lesions degrade recent learning but not temporally remote learning? Why do cortical lesions degrade temporally remote learning, and cause amnesia, but not recent or post-lesion learning? How is temporally graded amnesia caused by ablation of medial prefrontal cortex? How are mechanisms of motivated attention and the emergent state of consciousness linked during conditioning? How do neurotrophins, notably Brain Derived Neurotrophic Factor (BDNF), influence memory formation and consolidation? A neural model, called neurotrophic START, or nSTART, proposes answers to these questions. The nSTART model synthesizes and extends key principles, mechanisms, and properties of three previously published brain models of normal behavior. These three models describe aspects of how the brain can learn to categorize objects and events in the world; how the brain can learn the emotional meanings of such events, notably rewarding and punishing events, through cognitive-emotional interactions; and how the brain can learn to adaptively time attention paid to motivationally important events, and when to respond to these events, in a context-appropriate manner. The model clarifies how hippocampal adaptive timing mechanisms and BDNF may bridge the gap between stimuli during trace conditioning and thereby allow thalamocortical and corticocortical learning to take place and be consolidated. The simulated data arise as emergent properties of several brain regions interacting together. The model overcomes problems of alternative memory models, notably models wherein memories that are initially stored in hippocampus move to the neocortex during consolidation

A neural network model of normal and abnormal learning and memory consolidation

The amygdala and hippocampus interact with thalamocortical systems to regulate cognitive-emotional learning, and lesions of amygdala, hippocampus, thalamus, and cortex have different effects depending on the phase of learning when they occur. In examining eyeblink conditioning data, several questions arise: Why is the hippocampus needed for trace conditioning where there is a temporal gap between the conditioned stimulus offset and the onset of the unconditioned stimulus, but not needed for delay conditioning where stimuli temporally overlap and co-terminate? Why do amygdala lesions made before or immediately after training decelerate conditioning while those made later have no impact on conditioned behavior? Why do thalamic lesions degrade trace conditioning more than delay conditioning? Why do hippocampal lesions degrade recent learning but not temporally remote learning? Why do cortical lesions degrade temporally remote learning, and cause amnesia, but not recent or post-lesion learning? How is temporally graded amnesia caused by ablation of medial prefrontal cortex? How are mechanisms of motivated attention and the emergent state of consciousness linked during conditioning? How do neurotrophins, notably Brain Derived Neurotrophic Factor (BDNF), influence memory formation and consolidation? A neural model, called neurotrophic START, or nSTART, proposes answers to these questions. The nSTART model synthesizes and extends key principles, mechanisms, and properties of three previously published brain models of normal behavior. These three models describe aspects of how the brain can learn to categorize objects and events in the world; how the brain can learn the emotional meanings of such events, notably rewarding and punishing events, through cognitive-emotional interactions; and how the brain can learn to adaptively time attention paid to motivationally important events, and when to respond to these events, in a context-appropriate manner. The model clarifies how hippocampal adaptive timing mechanisms and BDNF may bridge the gap between stimuli during trace conditioning and thereby allow thalamocortical and corticocortical learning to take place and be consolidated. The simulated data arise as emergent properties of several brain regions interacting together. The model overcomes problems of alternative memory models, notably models wherein memories that are initially stored in hippocampus move to the neocortex during consolidation

Task-based fMRI investigation of the newborn brain: sensorimotor development and learning

Human brain development relies upon the interaction between genetic and environmental factors, and the latter plays a critical role during the perinatal period. In this period, neuronal plasticity through experience-dependent activity is enhanced in the sensory systems, and drive the maturation of the brain. While plasticity is essential for maturation, it is also a source of vulnerability as altered early experiences may interact with the normal course of development. This is particularly evident in infants born preterm, who are prematurely exposed to a sensory-rich environment, and at risk or neurodevelopmental disorders. In keeping with the somatosensory system being at a critical period for development during late gestation, sensorimotor disorders, such as cerebral palsy, are more common in preterm compared with full-term born infants. It is therefore important to understand the normal trajectory of sensorimotor development and how this may be moulded by early sensory experiences. It is well acknowledged that the sensorimotor cortex is topographically organised so that different body parts map to a specific location within the cortex and this map is generally referred to as the ``homunculus". Although the somatotopy has been well characterised in the mature brain, it remains unknown when this organisation emerges during development. Animal studies hints that functional cortical maps might emerge across the equivalent period to the third trimester of human gestation, nevertheless there is currently no evidence. Therefore, I first investigated the topography of the preterm somatosensory cortex in a group of newborn infants. In this purpose I used fMRI and automated robotic tools and measured the functional responses to different sensory simulations (delivered to the mouth, wrists and ankles). The results provide evidence that it is possible to identify distinct areas in the somatosensory cortex devoted to different body parts even in the preterm brain supporting the presence of an immature \textit{homunculus}. Next, I wanted to investigate how activity and development in the sensorimotor system are influenced by experience. Experience-dependent plasticity is the basis of learning (e.g. adaptive behaviour), which is observed in newborn infants. Associative learning in particular has been widely investigated in infants, however, the underlining neuronal processes have previously been poorly understood. To study the neural correlates of associative learning in newborn infants, I developed and used a classical conditioning paradigm in combination with robot-assisted fMRI. The results confirm that associative learning can occur even at this early stage of life and with non-aversive stimuli. More importantly, I could observe learning-induced changes in brain activity within the primary sensory cortices, suggesting that such experience can shape cortical circuitry and is likely to influence early brain development.Open Acces

Exploring model-based and model-free reinforcement learning in obsessive-compulsive disorder

RESUMO: A Perturbação Obsessivo-Compulsiva (POC) é uma doença neuropsiquiátrica comum, grave e incapacitante, para a qual os tratamentos actuais são ineficazes num grande número de casos. O instrumento mais utilizado para avaliar a gravidade de sintomas obsessivo-compulsivos é a Yale-Brown Obsessive-Compulsive Scale (YBOCS), que foi recentemente revista (Y-BOCS-II). No entanto, a sua validade de construto (tanto divergente como convergente) tem sido reportada como moderada e a sua validade de critério para diagnóstico de POC nunca foi testada. No primeiro capítulo desta tese testei, pela primeira vez, a validade de critério da Y-BOCS-II e demonstrei que um ponto de corte de 13 (pontuação total) atinge o melhor balanço entre sensibilidade e especificidade para o diagnóstico de POC. No entanto, confirmei que a sua validade divergente está longe de ser excelente. Este último achado levoume a procurar outros potenciais marcadores de POC. Têm sido demonstradas várias anomalias em doentes com POC utilizando tarefas neuropsicológicas ou técnicas de neuroimagem. Contudo, não existe ainda um marcador consistente para esta perturbação, que seja capaz de discriminar eficazmente pacientes que sofrem de POC, que seja sensível à mudança após intervenções terapêuticas e para o qual seja possível estabelecer uma correspondência com circuitos ou função cerebral. Uma abordagem que tem sido seguida nos últimos anos considera a POC como sendo caracterizada por uma disfunção nos sistemas cerebrais responsáveis pela aprendizagem de acções. As tarefas de decisão sequencial emergiram recentemente como um instrumento importante e sofisticado para estudar a aprendizagem de acções em humanos através da abordagem de reinforcement learning (RL). De acordo com a teoria subjacente ao RL, as acções podem ser aprendidas de duas formas distintas: um sistema modelbased funciona através da construção de um modelo interno das dinâmicas do ambiente e utiliza esse modelo para planear trajectórias comportamentais futuras, por oposição a um sistema model-free, que funciona armazenando o valor estimado das acções que foram implementadas recentemente e actualizando essas estimativas por tentativa e erro. As chamadas tarefas de decisão sequencial têm vindo a ser utilizadas para estabelecer associações entre disfunção de sistemas cerebrais de RL e algumas perturbações neuropsiquiátricas, como a POC, sendo que um desequilíbrio entre os sistemas model-based e model-free tem sido descrito. Através da aplicação de uma dessas tarefas de decisão sequencial, a two-step task, existe evidência que sugere que os doentes com POC têm um défice no sistema model-based. No entanto, neste paradigma em particular, antes de desempenhar esta tarefa os indivíduos recebem informação detalhada sobre a estrutura da mesma. Assim, não é claro como os dois principais sistemas de RL interagem quando os indivíduos aprendem exclusivamente através de interacção com o ambiente e como a informação explícita afecta as estratégias de RL. No segundo capítulo desta tese, desenvolvi uma nova tarefa de decisões sequenciais que permite não só quantificar o uso de estratégias modelbased RL e model-free RL, mas também diferenciar entre o impacto do conhecimento explícito da estrutura da tarefa e o impacto da experiência na mesma. Os resultados da aplicação da tarefa em indivíduos saudáveis demonstram que inicialmente a escolha de acções é controlada por aprendizagem model-free, com a aprendizagem model-based emergindo apenas numa minoria de indivíduos depois de experiência significativa com a tarefa, não emergindo de todo em indivíduos com POC, que por sua vez mostraram tendência para aumentar o uso de model-free RL com a experiência. Quando foi dada informação explícita sobre a estrutura da tarefa, observou-se um aumento dramático do uso de aprendizagem model-based, tanto nos voluntários saudáveis como em ambos os grupos clínicos. A informação explícita diminuiu o uso do sistema de aprendizagem model-free nos voluntários saudáveis e nos pacientes com perturbação do humor e ansiedade, mas essa diminuição não foi estatisticamente significativa no grupo de doentes com POC. Para além disso, depois das instruções, verificou-se em todos os grupos que a actualização do valor das acções aprendidas através do sistema model-free passou a ser mais influenciada pelo valor dos estados atingidos e menos influenciada pela consequência dos ensaios. Outro efeito da informação explícita sobre a estrutura da tarefa nos indivíduos saudáveis foi tornar as escolhas mais perseverantes, o que é consistente com uma modificação da estratégia de exploração. Estes resultados ajudam a clarificar o perfil de utilização de estratégias de RL dos pacientes com POC, que apresentam défice inespecíficos de aprendizagem model-based e achados mais específicos de maior uso de aprendizagem model-free, em ambos os casos antes de obterem informação sobrea estrutura da tarefa. Por fim, como a literatura ainda não é consensual sobre a interação entre um eventual sistema de model-based RL e um sistema de model-free RL nos circuitos cerebrais em humanos, devenvolvi um protocolo de ressonância magnética funcional para avaliar a escolha de ação sequencial com e sem instruções. Os resultados preliminares, em indivíduos saudáveis, sugerem que a reduced two-step task permite separar comportamento que utiliza aprendizagem predominantemente model-free (antes das instruções) de comportamento que utiliza aprendizagem predominantemente model-based (após as instruções), no mesmo indivíduo, estrutura da tarefa e ambiente. A análise dos dados de imagem funcional sugere que o conhecimento explícito sobre a estrutura da tarefa modifica a atividade neuronal no córtex paracingulado (cortex prefrontal medial) durante a transição do primeiro para o segundo passo da tarefa. Objectivos futuros incluem o uso de técnicas de análise multivariada para explorar a representação cerebral dos estados da tarefa e a aplicação deste protocolo de ressonância magnética funcional em populações clínicas.ABSTRACT: Obsessive-compulsive disorder (OCD) is a common, chronic and disabling neuropsychiatric condition for which current treatments are ineffective in a large proportion of cases. The gold-standard instrument to assess the severity of OCD symptoms is the Yale-Brown Obsessive-Compulsive Scale (Y-BOCS), which was recently revised (Y-BOCS-II). However, its construct validity has been reported has moderate and its criterion-related validity for the diagnosis of OCD has never been tested. In the first chapter of this dissertation, I tested, for the first time, criterion-related validity of the Y-BOCS-II and demonstrated that a cut-off of 13 (total score) attains the best balance between sensitivity and specificity for the diagnosis of OCD. However, I confirmed that its divergent validity is far from excellent. This last finding led me to search for other potential markers of OCD. Several abnormalities have been demonstrated in OCD patients in studies using neuropsychological and neuroimaging approaches, but we still lack a consistent marker for the disorder which is able to discriminate patients with OCD from healthy subjects or from patients with other mental disorders, which is sensitive to treatmentinduced changes, and which can be mapped to brain circuits or function. An approach which has been followed over the last decade is considering OCD as a disorder of action learning systems of the brain. Sequential decision tasks have recently emerged as an influential and sophisticated tool to investigate action learning in humans through the reinforcement learning (RL) framework. According to the RL framework, actions can be learned in two different ways: model-based control works by learning a model of the dynamics of the environment and later using that model to plan future behavioral trajectories, while model-free control works by storing the estimated value of recently taken actions and updating these estimates by trial-and-error. Sequential decision tasks have been used to assess associations between dysfunction in RL control systems and certain behavioral disorders, such as OCD, where an unbalance between model-based and model-free RL has been hypothesized. In fact, using the most commonly applied sequential decision task, the two-step task, evidence has been produced suggesting that OCD patients have a deficit in model-based learning. However, in this specific paradigm, subjects typically receive detailed information about task structure prior to performing the task. Thus, it remains unclear how different RL systems contribute when subjects learn exclusively from experience, and how explicit information about task structure modifies RL strategy. To address these questions, I created a sequential decision task requiring minimal prior instruction, the reduced two-step task. I assessed performance both prior to and after delivering explicit information on task structure, in healthy volunteers, patients with OCD and patients with other mood and anxiety disorders. Initially model-free control dominated, with model-based control emerging only in a minority of subjects after significant task experience, and not at all in patients with OCD, who had instead a tendency to increase their use of model-free control. Once explicit information about task structure was provided, a dramatic increase in the use of model-based RL was observed,similarly across healthy volunteers and both patient groups, including OCD. The debriefing also significantly decreased the use of model-free RL in healthy volunteers and in patients with mood and anxiety disorders, but not in OCD patients. Additionally, after instructions, model-free action value updates were influenced more by state values and less by trial outcomes, in all groups, and subject choices became more perseverative in healthy subjects, consistent with changes in exploration strategy. These results help in clarifying the RL profile for patients with OCD, with unspecific findings of deficient model-based control, and more specific findings of enhanced model-free control, in both cases prior to information about task structure. Finally, as the literature is not yet consensual on how model-free and modelbased RL systems interact in human brain circuits, I developed a functional magnetic resonance imaging (fMRI) protocol to assess uninstructed and instructed sequential action choice. Preliminary results in healthy subjects suggest that the fMRI version of the reduced two-step task allows to separate predominantly model-free control (before instructions) from predominantly model-based control (after instructions), in the same subject, task structure and environment. Across all sessions, choice events were associated with increases blood-oxygen-level-dependent (BOLD) activity in the left precentral gyrus and reward events were associated with increased BOLD activity in the ventral striatum. I found that explicit knowledge about task structure modifies blood-oxygen-level-dependent (BOLD) activity in the paracingulate cortex (medial prefrontal cortex) during the transition from the first- to the second-step of the task. Future directions include using multivariate pattern analysis techniques to explore how the brain represents state space in sequential decision tasks and applying the current fMRI protocol in clinical populations

Why laterality matters in trauma: sinister aspects of memory and emotion

This thesis presents an eclectic mix of studies which consider laterality in the context of previous findings of increased prevalence of Posttraumatic Stress Disorder (PTSD) in male combat veterans with non-consistent right hand preference. Two studies extend these findings not just to civilian populations and women, but to left handers and find that left, rather than mixed, handedness is associated with increased prevalence of PTSD in both general population and clinical samples, and to severity of symptoms in the former. To examine issues relevant to the fear response in healthy populations, a movie excerpt is shown to be theoretically likely to target the emotion of fear and to generate subjective and physiological (skin conductance) responses of fear. The film is used as a laboratory analogue of fear to examine possible differences in left and right handers in memory (for events of the film) and in an emotional Stroop paradigm known to produce a robust and large effect specifically in PTSD. According to predictions based on lateralisation of functions in the brain relevant to the fear response, left handers show a pattern of enhanced memory for visual items and poorer memory for verbal material compared to right handers. Immediately after viewing the film, left handers show an interference effect on the Stroop paradigm to general threat and film words and increased response latency compared to right handers, approaching performance of previously reported clinical samples with PTSD. A novel non-word Stroop task fails to show these effects, consistent both with accounts of interference as language processing effects and compromised verbal processing in PTSD. Unexpected inferior performance of females in memory for the film, contrary to previous literature, may also be amenable to explanations invoking compromised left hemisphere language functions in fear situations. In testing one theory of left handedness as due to increased levels of in utero testosterone, the 2D:4D (second to fourth digit ratio) provides mixed evidence in two samples. A possible association of more female-like digit ratios in males with PTSD is a tentative finding possibly relevant to sex differences in prevalence of PTSD. A critique of existing and inadequate theoretical accounts of handedness concludes the thesis and proposes a modification of the birth stress hypothesis to one specifically considering peri-natal trauma to account for the above findings. This hypothesis remains to be empirically tested

Anxiety and how to control it: the functional role of the bed nucleus of the stria terminalis.

Anxiety disorders afflict up to one third of the population. Research to date has primarily focused on the amygdala, however, new perspectives suggest that a tiny basal forebrain region known as the bed nucleus of the stria terminalis (BNST) may hold key insights into understanding and treating anxiety disorders. Therefore, my first aim was to empirically investigate the importance and influence of the BNST in anxiety processing. Using fearful faces and human screams as aversive stimuli, two threat conditions were created: one in which threats were certain and predictable (fear) and another in which threats were uncertain and unpredictable (anxiety). Results indicated that the amygdala showed preferential engagement during fear and displayed functional connectivity with regions involved in stimulus processing and motor response. By contrast, the BNST preferentially responded during anxiety and exhibited functional connectivity with prefrontal regions underlying interoception and rumination. Together, this suggests that the amygdala and BNST play distinct but complementary roles during threat processing, with the BNST specializing in the detection of potential threats to promote hypervigilant monitoring. A primary mechanism of impaired functioning in anxiety disorders is emotion dysregulation, and has been another key focus in research. However, most emotion regulation (ER) paradigms use explicitly cued pictorial stimuli (negative scenes or faces) that induce disgust, when anxiety, by definition, is a sustained response to uncertain or unpredictable prospective threats. Therefore, my second aim was to specifically investigate anxiety regulation. 30 participants underwent high-resolution fMRI (1.5 mm3) while performing a novel task — a hybrid of the previous task and a canonical ER task – in order to: 1) investigate whether the BNST can be downregulated during uncertain anticipation, and 2) characterize the prefrontal regulatory mechanisms. Results showed that anxiety regulation was associated with pronounced BNST downregulation, enhanced activation of prefrontal regions (right middle frontal gyrus [rMFG], right inferior frontal gyrus [rIFG]), and increased connectivity between the rIFG and BNST while simultaneously decreasing connectivity among attentional circuits. These results provide the first evidence that the BNST can be volitionally downregulated and further suggest that anxiety regulation modulates higher-order attentional systems to putatively reduce vigilance