Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

An International Laboratory for Systems and Computational Neuroscience

The neural basis of decision-making has been elusive and involves the coordinated activity of multiple brain structures. This NeuroView, by the International Brain Laboratory (IBL), discusses their efforts to develop a standardized mouse decision-making behavior, to make coordinated measurements of neural activity across the mouse brain, and to use theory and analyses to uncover the neural computations that support decision-making. The neural basis of decision-making has been elusive and involves the coordinated activity of multiple brain structures. This NeuroView, by the International Brain Laboratory (IBL), discusses their efforts to develop a standardized mouse decision-making behavior, to make coordinated measurements of neural activity across the mouse brain, and to use theory and analyses to uncover the neural computations that support decision-making

Can we identify non-stationary dynamics of trial-to-trial variability?"

Identifying sources of the apparent variability in non-stationary scenarios is a fundamental problem in many biological data analysis settings. For instance, neurophysiological responses to the same task often vary from each repetition of the same experiment (trial) to the next. The origin and functional role of this observed variability is one of the fundamental questions in neuroscience. The nature of such trial-to-trial dynamics however remains largely elusive to current data analysis approaches. A range of strategies have been proposed in modalities such as electro-encephalography but gaining a fundamental insight into latent sources of trial-to-trial variability in neural recordings is still a major challenge. In this paper, we present a proof-of-concept study to the analysis of trial-to-trial variability dynamics founded on non-autonomous dynamical systems. At this initial stage, we evaluate the capacity of a simple statistic based on the behaviour of trajectories in classification settings, the trajectory coherence, in order to identify trial-to-trial dynamics. First, we derive the conditions leading to observable changes in datasets generated by a compact dynamical system (the Duffing equation). This canonical system plays the role of a ubiquitous model of non-stationary supervised classification problems. Second, we estimate the coherence of class-trajectories in empirically reconstructed space of system states. We show how this analysis can discern variations attributable to non-autonomous deterministic processes from stochastic fluctuations. The analyses are benchmarked using simulated and two different real datasets which have been shown to exhibit attractor dynamics. As an illustrative example, we focused on the analysis of the rat's frontal cortex ensemble dynamics during a decision-making task. Results suggest that, in line with recent hypotheses, rather than internal noise, it is the deterministic trend which most likely underlies the observed trial-to-trial variability. Thus, the empirical tool developed within this study potentially allows us to infer the source of variability in in-vivo neural recordings

Spatial Intuition in Elementary Arithmetic: A Neurocomputational Account

Elementary arithmetic (e.g., addition, subtraction) in humans has been shown to exhibit spatial properties. Its exact nature has remained elusive, however. To address this issue, we combine two earlier models for parietal cortex: A model we recently proposed on number-space interactions and a modeling framework of parietal cortex that implements radial basis functions for performing spatial transformations. Together, they provide us with a framework in which elementary arithmetic is based on evolutionarily more basic spatial transformations, thus providing the first implemented instance of Dehaene and Cohen's recycling hypothesis

Top-Down Feedback in an HMAX-Like Cortical Model of Object Perception Based on Hierarchical Bayesian Networks and Belief Propagation

PubMed ID: 2313976

Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows (“explaining away”) and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons

Surveillance of Sentinel Node-Positive Melanoma Patients with Reasons for Exclusion from MSLT-II:Multi-Institutional Propensity Score Matched Analysis

BACKGROUND: In sentinel lymph node (SLN)-positive melanoma, two randomized trials demonstrated equivalent melanoma-specific survival with nodal surveillance vs completion lymph node dissection (CLND). Patients with microsatellites, extranodal extension (ENE) in the SLN, or >3 positive SLNs constitute a high-risk group largely excluded from the randomized trials, for whom appropriate management remains unknown. STUDY DESIGN: SLN-positive patients with any of the three high-risk features were identified from an international cohort. CLND patients were matched 1:1 with surveillance patients using propensity scores. Risk of any-site recurrence, SLN-basin-only recurrence, and melanoma-specific mortality were compared. RESULTS: Among 1,154 SLN-positive patients, 166 had ENE, microsatellites, and/or >3 positive SLN. At 18.5 months median follow-up, 49% had recurrence (vs 26% in patients without high-risk features, p 3 positive SLN constitute a high-risk group with a 2-fold greater recurrence risk. For those managed with nodal surveillance, SLN-basin recurrences were more frequent, but all-site recurrence and melanoma-specific mortality were comparable to patients treated with CLND. Most recurrences were outside the SLN-basin, supporting use of nodal surveillance for SLN-positive patients with microsatellites, ENE, and/ or >3 positive SLN

Happiness matters : exploring the linkages between personality, personal happiness, and work-related psychological health among priests and sisters in Italy

This study responds to the challenge posed by Rossetti’s work to explore the antecedents and consequences of individual differences in happiness among priests and religious sisters. The Oxford Happiness Questionnaire was completed together with measures of personality and work-related psychological health by 95 priests and 61 religious sisters. Overall the data demonstrated high levels of personal happiness among priests and religious sisters, but also significant signs of vulnerability. Personality provided significant prediction of individual differences in both personal happiness and work-related psychological health. However, personal happiness provided additional protection against work-related emotional exhaustion and additional enhancement of work-related satisfaction. These findings suggest that acknowledging and affirming personal happiness may enhance the work-related psychological health of Catholic priests and religious sisters