Multivariate decoding of brain images using ordinal regression.

Neuroimaging data are increasingly being used to predict potential outcomes or groupings, such as clinical severity, drug dose response, and transitional illness states. In these examples, the variable (target) we want to predict is ordinal in nature. Conventional classification schemes assume that the targets are nominal and hence ignore their ranked nature, whereas parametric and/or non-parametric regression models enforce a metric notion of distance between classes. Here, we propose a novel, alternative multivariate approach that overcomes these limitations - whole brain probabilistic ordinal regression using a Gaussian process framework. We applied this technique to two data sets of pharmacological neuroimaging data from healthy volunteers. The first study was designed to investigate the effect of ketamine on brain activity and its subsequent modulation with two compounds - lamotrigine and risperidone. The second study investigates the effect of scopolamine on cerebral blood flow and its modulation using donepezil. We compared ordinal regression to multi-class classification schemes and metric regression. Considering the modulation of ketamine with lamotrigine, we found that ordinal regression significantly outperformed multi-class classification and metric regression in terms of accuracy and mean absolute error. However, for risperidone ordinal regression significantly outperformed metric regression but performed similarly to multi-class classification both in terms of accuracy and mean absolute error. For the scopolamine data set, ordinal regression was found to outperform both multi-class and metric regression techniques considering the regional cerebral blood flow in the anterior cingulate cortex. Ordinal regression was thus the only method that performed well in all cases. Our results indicate the potential of an ordinal regression approach for neuroimaging data while providing a fully probabilistic framework with elegant approaches for model selection

Outcome contingency selectively affects the neural coding of outcomes but not of tasks

Value-based decision-making is ubiquitous in every-day life, and critically depends on the contingency between choices and their outcomes. Only if outcomes are contingent on our choices can we make meaningful value-based decisions. Here, we investigate the effect of outcome contingency on the neural coding of rewards and tasks. Participants performed a reversal-learning paradigm in which reward outcomes were contingent on trial-by-trial choices, and performed a ‘free choice’ paradigm in which rewards were random and not contingent on choices. We hypothesized that contingent outcomes enhance the neural coding of rewards and tasks, which was tested using multivariate pattern analysis of fMRI data. Reward outcomes were encoded in a large network including the striatum, dmPFC and parietal cortex, and these representations were indeed amplified for contingent rewards. Tasks were encoded in the dmPFC at the time of decision-making, and in parietal cortex in a subsequent maintenance phase. We found no evidence for contingency-dependent modulations of task signals, demonstrating highly similar coding across contingency conditions. Our findings suggest selective effects of contingency on reward coding only, and further highlight the role of dmPFC and parietal cortex in value-based decision-making, as these were the only regions strongly involved in both reward and task coding

Visual consciousness tracked with direct intracranial recording from early visual cortex in humans

A fundamental question in cognitive neuroscience is how neuronal representations are related to conscious experience.
Two key questions are: where in the brain such representations are located, and at what point in time they correlate with conscious experience. In line with this issue, a hotly debated question is whether primary visual cortex (V1) contributes to visual consciousness, or whether this depends only on higher-order cortices. Here we investigated this issue by recording directly from early visual cortex in two neurosurgical patients undergoing epilepsy monitoring with intracranial electrocorticogram (ECoG) electrodes that covered early visual cortices, including the dorsal and ventral banks of the calcarine sulcus. We used Continuous Flash Suppression (CFS)to investigate the time course of when ‘invisible’ stimuli broke interocular suppression. Participants were asked to watch faces presented under CFS, to push a button when they started to see any part of the face, and then to indicate its spatial location. This occurred over several seconds. During the task performance we recorded intracranial ECoG at high spatiotemporal resolution from all contacts in parallel. We used multivariate decoding techniques and found that the location of the invisible face stimulus became decodable from neuronal activity 1.8 sec before the subject’s button press. Counter-intuitively, the same cortical sites from which we were able to decode this predictive signal showed a decrease in activity immediately prior to the transition from invisibility to visibility. Furthermore, we observed an increase in coherence among widely separated electrodes during the invisible epoch, which collapsed to a focal ensemble when the stimulus became visible. These results suggest that diffuse coherent representation is insufficient for visual awareness and that locally specialized patterns of activation may be key to consciousness. Our findings are consistent with one recently proposed framework for understanding consciousness utilizing information integration theory (Tononi, 2008)

Neural overlap of L1 and L2 semantic representations across visual and auditory modalities : a decoding approach/

This study investigated whether brain activity in Dutch-French bilinguals during semantic access to concepts from one language could be used to predict neural activation during access to the same concepts from another language, in different language modalities/tasks. This was tested using multi-voxel pattern analysis (MVPA), within and across language comprehension (word listening and word reading) and production (picture naming). It was possible to identify the picture or word named, read or heard in one language (e.g. maan, meaning moon) based on the brain activity in a distributed bilateral brain network while, respectively, naming, reading or listening to the picture or word in the other language (e.g. lune). The brain regions identified differed across tasks. During picture naming, brain activation in the occipital and temporal regions allowed concepts to be predicted across languages. During word listening and word reading, across-language predictions were observed in the rolandic operculum and several motor-related areas (pre- and postcentral, the cerebellum). In addition, across-language predictions during reading were identified in regions typically associated with semantic processing (left inferior frontal, middle temporal cortex, right cerebellum and precuneus) and visual processing (inferior and middle occipital regions and calcarine sulcus). Furthermore, across modalities and languages, the left lingual gyrus showed semantic overlap across production and word reading. These findings support the idea of at least partially language- and modality-independent semantic neural representations

Frontal alpha oscillations distinguish leaders from followers: Multivariate decoding of mutually interacting brains

Successful social interactions rely upon the abilities of two or more people to mutually exchange information in real-time, while simultaneously adapting to one another. The neural basis of social cognition has mostly been investigated in isolated individuals, and more recently using two-person paradigms to quantify the neuronal dynamics underlying social interaction. While several studies have shown the relevance of understanding complementary and mutually adaptive processes, the neural mechanisms underlying such coordinative behavioral patterns during joint action remain largely unknown. Here, we employed a synchronized finger-tapping task while measuring dual-EEG from pairs of human participants who either mutually adjusted to each other in an interactive task or followed a computer metronome. Neurophysiologically, the interactive condition was characterized by a stronger suppression of alpha and low-beta oscillations over motor and frontal areas in contrast to the non-interactive computer condition. A multivariate analysis of two-brain activity to classify interactive versus non-interactive trials revealed asymmetric patterns of the frontal alpha-suppression in each pair, during both task anticipation and execution, such that only one member showed the frontal component. Analysis of the behavioral data showed that this distinction coincided with the leader–follower relationship in 8/9 pairs, with the leaders characterized by the stronger frontal alpha-suppression. This suggests that leaders invest more resources in prospective planning and control. Hence our results show that the spontaneous emergence of leader–follower relationships in dyadic interactions can be predicted from EEG recordings of brain activity prior to and during interaction. Furthermore, this emphasizes the importance of investigating complementarity in joint action

Stubborn Predictions in Primary Visual Cortex

Perceivers can use past experiences to make sense of ambiguous sensory signals. However, this may be inappropriate when the world changes and past experiences no longer predict what the future holds. Optimal learning models propose that observers decide whether to stick with or update their predictions by tracking the uncertainty or "precision" of their expectations. However, contrasting theories of prediction have argued that we are prone to misestimate uncertainty-leading to stubborn predictions that are difficult to dislodge. To compare these possibilities, we had participants learn novel perceptual predictions before using fMRI to record visual brain activity when predictive contingencies were disrupted-meaning that previously "expected" events become objectively improbable. Multivariate pattern analyses revealed that expected events continued to be decoded with greater fidelity from primary visual cortex, despite marked changes in the statistical structure of the environment, which rendered these expectations no longer valid. These results suggest that our perceptual systems do indeed form stubborn predictions even from short periods of learning-and more generally suggest that top-down expectations have the potential to help or hinder perceptual inference in bounded minds like ours

Feature-reweighted representational similarity analysis: A method for improving the fit between computational models, brains, and behavior

Representational Similarity Analysis (RSA) has emerged as a popular method for relating representational spaces from human brain activity, behavioral data, and computational models. RSA is based on the comparison of representational (dis-)similarity matrices (RDM or RSM), which characterize the pairwise (dis-)similarities of all conditions across all features (e.g. fMRI voxels or units of a model). However, classical RSA treats each feature as equally important. This 'equal weights' assumption contrasts with the flexibility of multivariate decoding, which reweights individual features for predicting a target variable. As a consequence, classical RSA may lead researchers to underestimate the correspondence between a model and a brain region and, in case of model comparison, may lead them to select an inferior model. The aim of this work is twofold: First, we sought to broadly test feature-reweighted RSA (FR-RSA) applied to computational models and reveal the extent to which reweighting model features improves RSM correspondence and affects model selection. Previous work suggested that reweighting can improve model selection in RSA but it has remained unclear to what extent these results generalize across datasets and data modalities. To draw more general conclusions, we utilized a range of publicly available datasets and three popular deep neural networks (DNNs). Second, we propose voxel-reweighted RSA, a novel use case of FR-RSA that reweights fMRI voxels, mirroring the rationale of multivariate decoding of optimally combining voxel activity patterns. We found that reweighting individual model units markedly improved the fit between model RSMs and target RSMs derived from several fMRI and behavioral datasets and affected model selection, highlighting the importance of considering FR-RSA. For voxel-reweighted RSA, improvements in RSM correspondence were even more pronounced, demonstrating the utility of this novel approach. We additionally show that classical noise ceilings can be exceeded when FR-RSA is applied and propose an updated approach for their computation. Taken together, our results broadly validate the use of FR-RSA for improving the fit between computational models, brain, and behavioral data, possibly allowing us to better adjudicate between competing computational models. Further, our results suggest that FR-RSA applied to brain measurement channels could become an important new method to assess the correspondence between representational spaces

Frontoparietal representations of task context support the flexible control of goal-directed cognition.

Cognitive control allows stimulus-response processing to be aligned with internal goals and is thus central to intelligent, purposeful behavior. Control is thought to depend in part on the active representation of task information in prefrontal cortex (PFC), which provides a source of contextual bias on perception, decision making, and action. In the present study, we investigated the organization, influences, and consequences of context representation as human subjects performed a cued sorting task that required them to flexibly judge the relationship between pairs of multivalent stimuli. Using a connectivity-based parcellation of PFC and multivariate decoding analyses, we determined that context is specifically and transiently represented in a region spanning the inferior frontal sulcus during context-dependent decision making. We also found strong evidence that decision context is represented within the intraparietal sulcus, an area previously shown to be functionally networked with the inferior frontal sulcus at rest and during task performance. Rule-guided allocation of attention to different stimulus dimensions produced discriminable patterns of activation in visual cortex, providing a signature of top-down bias over perception. Furthermore, demands on cognitive control arising from the task structure modulated context representation, which was found to be strongest after a shift in task rules. When context representation in frontoparietal areas increased in strength, as measured by the discriminability of high-dimensional activation patterns, the bias on attended stimulus features was enhanced. These results provide novel evidence that illuminates the mechanisms by which humans flexibly guide behavior in complex environments

Neurocomputational mechanisms underlying effort-based value integration

In everyday life, we encounter many decisions requiring the consideration of prospective effort, such as taking exercise and altruistic behaviors. Therefore, the ability to accurately weigh effort costs against potential rewards is critical for optimal goal-directed behavior. The common currency theory proposes that values of different options are mapped to a common scale by a neural network to ensure efficient decision-making across different cost types. This theory provides a general framework to explain how rewards and costs are integrated and has gained popularity in decision-making associated with other types of cost, such as risk and delay. Although a few studies have examined the computational and neural mechanisms underlying effort-based value integration, it remains unclear if effort discounts prospective outcomes in a similar way to other costs, and, at the neural level, it is still under debate if effort-based value integration engages a general valuation neural network as suggested by the common currency theory, or instead relies on a specific network compared with other cost domains. In this dissertation, I address these questions across a meta-analytic study and two empirical studies. In study 1 (Lopez-Gamundi et al., 2021) of this dissertation, we conducted two separate meta-analyses to examine consistent neural correlates of effort-reward integration or raw effort requirement in related fMRI studies. We found that the vmPFC activity scaled positively with net value but negatively with raw effort. On the other hand, the dmPFC was also identified in both analyses, but its activity scaled negatively with net value and positively with raw effort. These findings are generally consistent with previous findings in other cost domains. In study 2 (Yao et al., 2022), to directly test if the common currency theory could be applied to value integration during effort-based decision-making, we reanalyzed the choice behavior and fMRI data of an open-access dataset, which included both effort-based and risky (one-option) decision-making tasks. Using computational modeling, we found that effort and risk showed distinct discounting effects on prospective outcomes. At the neural level, we conducted multivariate decoding analyses and found that a large cluster including both the vmPFC and dmPFC represented subjective value independent of cost types. In study 3 (Yao et al., 2022), we examined the replicability of the findings of Study 2 in an independent sample of participants. Moreover, to maintain similar overall acceptance rates between tasks, we estimated participant-specific indifference points for all combinations of rewards and costs (effort or risk) before scanning and manipulated the amounts of smaller rewards around these indifference points during scanning. We confirmed that effort and risk distinctively devalued rewards. At the neural level, we found that the dmPFC represented subjective value in a task-independent manner. Taken together, these findings highlight the role of the dmPFC in subjective value computation across effort-based and risky decision-making. Finally, I discuss how these results may reconcile the ongoing debates on the neural mechanisms underlying effort-reward integration and outline potential implications for the common currency theory.Im Alltag sind wir mit vielen Entscheidungen konfrontiert, die eine Abwägung des voraussichtlichen Aufwands erfordern, wie z. B. sportliche Aktivitäten und altruistisches Verhalten. Daher ist die Fähigkeit, die Kosten des Aufwands mit den potenziellen Belohnungen genau abzuwägen, entscheidend für optimales zielgerichtetes Verhalten. Die Theorie der gemeinsamen Währung besagt, dass die Werte verschiedener Optionen auf einer gemeinsamen Skala abgebildet werden, um eine effiziente Entscheidungsfindung über verschiedene Kostenarten hinweg zu gewährleisten. Diese Theorie bietet einen allgemeinen Rahmen, um zu erklären, wie Belohnungen und Kosten integriert werden, und hat bei der Entscheidungsfindung im Zusammenhang mit anderen Kostenarten wie Risiko und Verzögerung an Popularität gewonnen. Obwohl einige Studien die komputationalen und neuronalen Mechanismen untersucht haben, die der aufwandsbasierten Integration subjektiver Werte zugrunde liegen, bleibt unklar, ob Aufwand erwartete Ereignisse in ähnlicher Weise wie andere Kosten diskontiert. Weiterhin wird auf neuronaler Ebene diskutiert, ob die aufwandsbasierte Wertintegration auf ein allgemeines neuronales Bewertungsnetzwerk zurückzuführen ist, wie es die Theorie der gemeinsamen Währung nahelegt oder stattdessen auf ein spezifisches Netzwerk im Vergleich zu anderen Arten von Kosten zurückgreift. In dieser Dissertation befasse ich mich mit diesen Fragen im Rahmen einer Meta-Analyse früherer Studien und zweier empirischer Studien. In der ersten Dissertationsstudie (Lopez-Gamundi et al., 2021) haben wir zwei separate Meta-Analysen durchgeführt, um neuronale Korrelate der Anstrengungs-Belohnungs-Integration bzw. reiner Anstrengungsanforderung in verwandten fMRI-Studien zu untersuchen. Wir fanden heraus, dass die Aktivität des ventromedialen präfrontalen Kortex (vmPFC) positiv mit subjektiven Werten, aber negativ mit reinen Aufwandsanforderungen skaliert. Andererseits wurde der dorsomediale präfrontale Kortex (dmPFC) in beiden Analysen identifiziert, zeigte aber ein entgegengesetztes Aktivitätsmuster. Diese Ergebnisse stimmen im Allgemeinen mit früheren Erkenntnissen bei anderen Arten von Kosten überein. In Studie 2 (Yao et al., 2022) haben wir das Wahlverhalten und die Daten eines frei zugänglichen Datensatzes mittels funktioneller Magnetresonanztomografie (fMRT), der sowohl anstrengungsbasierte als auch riskante (eine Option) Entscheidungsaufgaben enthielt, erneut analysiert, um direkt zu prüfen, ob die Theorie der gemeinsamen Währung auf die Wertintegration bei anstrengungsbasierten Entscheidungen angewendet werden kann. Mithilfe von Computermodellen fanden wir heraus, dass Anstrengung und Risiko unterschiedliche Diskontierungseffekte auf prospektive Ergebnisse haben. Auf neuronaler Ebene führten wir multivariate Dekodierungsanalysen durch und fanden heraus, dass ein großes Cluster, das sowohl den vmPFC als auch den dmPFC umfasst, den subjektiven Wert unabhängig von der Art der Kosten repräsentiert. In Studie 3 (Yao et al., 2022) untersuchten wir die Replizierbarkeit der Ergebnisse aus Studie 2 mit einer unabhängigen Stichprobe von Versuchspersonen. Um ein ähnliches Entscheidungsverhalten zwischen den Aufgaben sicherzustellen, haben wir vor dem Scannen teilnehmerspezifische Indifferenzpunkte für alle Kombinationen von Belohnungen und Kosten (Aufwand oder Risiko) geschätzt. Zudem haben wir während des Scannens die Beträge kleinerer Belohnungen um diese Indifferenzpunkte manipuliert. Hierdurch konnten wir bestätigten, dass Aufwand und Risiko die Belohnungen deutlich abwerteten. Auf neuronaler Ebene stellten wir fest, dass der dmPFC den subjektiven Wert aufgabenunabhängig repräsentiert. Zusammengenommen betonen die Ergebnisse die Rolle des dmPFC bei der subjektiven Wertberechnung unter Anstrengung und Risiko. Abschließend erörtere ich, wie diese Ergebnisse die laufenden Debatten über die neuronalen Mechanismen der Integration von Aufwand und Belohnung in Einklang bringen können und skizziere mögliche Implikationen für die Theorie der gemeinsamen Währung