The Neural Basis of Following Advice

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people's learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this “outcome-bonus” accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model's outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice

Smoothness for Simultaneous Composition of Mechanisms with Admission

We study social welfare of learning outcomes in mechanisms with admission. In our repeated game there are $n$ bidders and $m$ mechanisms, and in each round each mechanism is available for each bidder only with a certain probability. Our scenario is an elementary case of simple mechanism design with incomplete information, where availabilities are bidder types. It captures natural applications in online markets with limited supply and can be used to model access of unreliable channels in wireless networks. If mechanisms satisfy a smoothness guarantee, existing results show that learning outcomes recover a significant fraction of the optimal social welfare. These approaches, however, have serious drawbacks in terms of plausibility and computational complexity. Also, the guarantees apply only when availabilities are stochastically independent among bidders. In contrast, we propose an alternative approach where each bidder uses a single no-regret learning algorithm and applies it in all rounds. This results in what we call availability-oblivious coarse correlated equilibria. It exponentially decreases the learning burden, simplifies implementation (e.g., as a method for channel access in wireless devices), and thereby addresses some of the concerns about Bayes-Nash equilibria and learning outcomes in Bayesian settings. Our main results are general composition theorems for smooth mechanisms when valuation functions of bidders are lattice-submodular. They rely on an interesting connection to the notion of correlation gap of submodular functions over product lattices.Comment: Full version of WINE 2016 pape

An interpretable approach for social network formation among heterogeneous agents

Understanding the mechanisms of network formation is central in social network analysis. Network formation has been studied in many research fields with their different focuses; for example, network embedding algorithms in machine learning literature consider broad heterogeneity among agents while the social sciences emphasize the interpretability of link formation mechanisms. Here we propose a social network formation model that integrates methods in multiple disciplines and retain both heterogeneity and interpretability. We represent each agent by an “endowment vector” that encapsulates their features and use game-theoretical methods to model the utility of link formation. After applying machine learning methods, we further analyze our model by examining micro- and macro- level properties of social networks as most agent-based models do. Our work contributes to the literature on network formation by combining the methods in game theory, agent-based modeling, machine learning, and computational sociology.King Abdulaziz City of Science and Technology (Saudia Arabia)MIT Trust Data Consortiu

Bayesian modelling captures inter-individual differences in social belief computations in the putamen and insula

Computational models of social learning and decision-making provide mechanistic tools to investigate the neural mechanisms that are involved in understanding other people. While most studies employ explicit instructions to learn from social cues, everyday life is characterized by the spontaneous use of such signals (e.g., the gaze of others) to infer on internal states such as intentions. To investigate the neural mechanisms of the impact of gaze cues on learning and decision-making, we acquired behavioural and fMRI data from 50 participants performing a probabilistic task, in which cards with varying winning probabilities had to be chosen. In addition, the task included a computer-generated face that gazed towards one of these cards providing implicit advice. Participants\u2019 individual belief trajectories were inferred using a hierarchical Gaussian filter (HGF) and used as predictors in a linear model of neuronal activation. During learning, social prediction errors were correlated with activity in inferior frontal gyrus and insula. During decision-making, the belief about the accuracy of the social cue was correlated with activity in inferior temporal gyrus, putamen and pallidum while the putamen and insula showed activity as a function of individual differences in weighting the social cue during decision-making. Our findings demonstrate that model-based fMRI can give insight into the behavioural and neural aspects of spontaneous social cue integration in learning and decision-making. They provide evidence for a mechanistic involvement of specific components of the basal ganglia in subserving these processes

The neuro-computational role of uncertainty in anxiety

Anxiety disorders are the most common mental health disorders and comprise a large number of years lost to disability. The work in this thesis is oriented towards understanding anxiety using a computational approach, focusing on uncertainty estimation as a key process. Chapter 1 introduces the role of uncertainty within anxiety and motivates the subsequent experimental chapters. Chapter 2 is a review of the computational role of the amygdala in humans, a key area for uncertainty computation. Chapter 3 is an experimental chapter which aimed to address gaps in the literature highlighted in the preceding chapters, namely the link between sensory uncertainty processing and anxiety and the role of the amygdala in this process. This chapter focuses on the development of a novel computational hierarchical Bayesian model to quantify sensory uncertainty and its application to neuroimaging data, with intolerance of uncertainty relating to greater neural activation in the insula but not amygdala. Chapter 4 targets the computational mechanisms underlying the negative self-bias observed in subclinical social anxiety. Again, this chapter focuses on the development of novel computational belief-update models which explicitly model uncertainty. Here, we see that a reduced trait self-positivity underpins this negative social evaluation process. The final experimental chapter presented in Chapter 5 investigates the link between different computational mechanisms, such as uncertainty, and a range of mood and anxiety symptomatology. This study revealed cognitive, social and somatic computational profiles that share a threat bias mechanism but have distinct negative-self bias and aversive learning signatures. Contrary to expectations, none of the uncertainty measures showed any associations with anxiety symptom subtypes. Finally, chapter 6 brings together the work in this thesis and alongside limitations of the work, discusses how these experiments contribute to our understanding of anxiety and the role of uncertainty across the anxiety spectrum

Vicarious Reinforcement Learning Signals When Instructing Others

Reinforcement learning (RL) theory posits that learning is driven by discrepancies between the predicted and actual outcomes of actions (prediction errors [PEs]). In social environments, learning is often guided by similar RL mechanisms. For example, teachers monitor the actions of students and provide feedback to them. This feedback evokes PEs in students that guide their learning. We report the first study that investigates the neural mechanisms that underpin RL signals in the brain of a teacher. Neurons in the anterior cingulate cortex (ACC) signal PEs when learning from the outcomes of one's own actions but also signal information when outcomes are received by others. Does a teacher's ACC signal PEs when monitoring a student's learning? Using fMRI, we studied brain activity in human subjects (teachers) as they taught a confederate (student) action–outcome associations by providing positive or negative feedback. We examined activity time-locked to the students' responses, when teachers infer student predictions and know actual outcomes. We fitted a RL-based computational model to the behavior of the student to characterize their learning, and examined whether a teacher's ACC signals when a student's predictions are wrong. In line with our hypothesis, activity in the teacher's ACC covaried with the PE values in the model. Additionally, activity in the teacher's insula and ventromedial prefrontal cortex covaried with the predicted value according to the student. Our findings highlight that the ACC signals PEs vicariously for others' erroneous predictions, when monitoring and instructing their learning. These results suggest that RL mechanisms, processed vicariously, may underpin and facilitate teaching behaviors

The mechanics of CSCL macro scripts

Macro scripts structure collaborative learning and foster the emergence of knowledge-productive interactions such as argumentation, explanations and mutual regulation. We propose a pedagogical model for the designing of scripts and illustrate this model using three scripts. In brief, a script disturbs the natural convergence of a team and in doing so increases the intensity of interaction required between team members for the completion of their collaborative task. The nature of the perturbation determines the types of interactions that are necessary for overcoming it: for instance, if a script provides students with conflicting evidence, more argumentation is required before students can reach an agreement. Tools for authoring scripts manipulate abstract representations of the script components and the mechanisms that relate components to one another. These mechanisms are encompassed in the transformation of data structures (social structure, resources structure and products structure) between script phases. We describe how this pedagogical design model is translated into computational structures in three illustrated script

Associations between aversive learning processes and transdiagnostic psychiatric symptoms revealed by large-scale phenotyping

Background: Aversive learning processes are a candidate source of dysfunction in psychiatric disorders. Here symptom expression in a range of conditions is linked to altered threat perception, manifesting particularly in uncertain environments. How precise computational mechanisms that support aversive learning, and uncertainty estimation, relate to the presence of specific psychiatric symptoms remains undetermined. Methods: 400 subjects completed a novel online game-based aversive learning task, requiring avoidance of negative outcomes, in conjunction with completing measures of common psychiatric symptoms. We used a probabilistic computational model to measure distinct processes involved in learning, in addition to inferred estimates of safety likelihood and uncertainty. We tested for associations between learning processes and traditional psychiatric constructs alongside transdiagnostic factors using linear models. We used partial least squares regression to identify components of psychopathology grounded in both aversive learning behaviour and symptom self-report. Results: State anxiety and a transdiagnostic compulsivity-related factor were associated with enhanced learning from safety. However, data-driven analysis using partial least squares regression indicated the presence of two separable components across our behavioural and questionnaire data: one linked enhanced safety learning and lower estimated uncertainty to physiological anxiety, compulsivity, and impulsivity; the other linked enhanced threat learning and heightened uncertainty estimation to symptoms of depression and social anxiety. Conclusions: Our findings implicate aversive learning processes under uncertainty to the expression of psychiatric symptoms that cut across traditional diagnostic boundaries. These relationships are more complex than previously conceptualised. Future research should focus on understanding the neural mechanisms underlying alterations in aversive learning and how these lead to the development of symptoms and disorder

The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning

While there is no doubt that social signals affect human reinforcement learning, there is still no consensus about how this process is computationally implemented. To address this issue, we compared three psychologically plausible hypotheses about the algorithmic implementation of imitation in reinforcement learning. The first hypothesis, decision biasing (DB), postulates that imitation consists in transiently biasing the learner's action selection without affecting their value function. According to the second hypothesis, model-based imitation (MB), the learner infers the demonstrator's value function through inverse reinforcement learning and uses it to bias action selection. Finally, according to the third hypothesis, value shaping (VS), the demonstrator's actions directly affect the learner's value function. We tested these three hypotheses in 2 experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task. We show through model comparison and model simulation that VS provides the best explanation of learner's behavior. Results replicated in a third independent experiment featuring a larger cohort and a different design (N = 302). In our experiments, we also manipulated the quality of the demonstrators' choices and found that learners were able to adapt their imitation rate, so that only skilled demonstrators were imitated. We proposed and tested an efficient meta-learning process to account for this effect, where imitation is regulated by the agreement between the learner and the demonstrator. In sum, our findings provide new insights and perspectives on the computational mechanisms underlying adaptive imitation in human reinforcement learning

CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation

Dating and romantic relationships not only play a huge role in our personal lives but also collectively influence and shape society. Today, many romantic partnerships originate from the Internet, signifying the importance of technology and the web in modern dating. In this paper, we present a text-based computational approach for estimating the relationship compatibility of two users on social media. Unlike many previous works that propose reciprocal recommender systems for online dating websites, we devise a distant supervision heuristic to obtain real world couples from social platforms such as Twitter. Our approach, the CoupleNet is an end-to-end deep learning based estimator that analyzes the social profiles of two users and subsequently performs a similarity match between the users. Intuitively, our approach performs both user profiling and match-making within a unified end-to-end framework. CoupleNet utilizes hierarchical recurrent neural models for learning representations of user profiles and subsequently coupled attention mechanisms to fuse information aggregated from two users. To the best of our knowledge, our approach is the first data-driven deep learning approach for our novel relationship recommendation problem. We benchmark our CoupleNet against several machine learning and deep learning baselines. Experimental results show that our approach outperforms all approaches significantly in terms of precision. Qualitative analysis shows that our model is capable of also producing explainable results to users.Comment: Accepted at ICWSM 201