Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

Thermodynamics of a class of non-asymptotically flat black holes in Einstein-Maxwell-Dilaton theory

We analyse in detail the thermodynamics in the canonical and grand canonical ensembles of a class of non-asymptotically flat black holes of the Einstein-(anti) Maxwell-(anti) Dilaton theory in 4D with spherical symmetry. We present the first law of thermodynamics, the thermodynamic analysis of the system through the geometrothermodynamics methods, Weinhold, Ruppeiner, Liu-Lu-Luo-Shao and the most common, that made by the specific heat. The geometric methods show a curvature scalar identically zero, which is incompatible with the results of the analysis made by the non null specific heat, which shows that the system is thermodynamically interacting, does not possess extreme case nor phase transition. We also analyse the local and global stability of the thermodynamic system, and obtain a local and global stability for the normal case for 0<\gamma<1 and for other values of \gamma, an unstable system. The solution where \gamma=0 separates the class of locally and globally stable solutions from the unstable ones.Comment: 18 pages, version accepted for publication in General Relativity and Gravitatio

The Nexus Solutions Tool (NEST): An open platform for optimizing multi-scale energy-water-land system transformations

The energy-water-land nexus represents a critical leverage future policies must draw upon to reduce trade-offs between sustainable development objectives. Yet, existing long-term planning tools do not provide the scope or level of integration across the nexus to unravel important development constraints. Moreover, existing tools and data are not always made openly available or are implemented across disparate modeling platforms that can be difficult to link directly with modern scientific computing tools and databases. In this paper, we present the Nexus Solutions Tool (NEST): a new open modeling platform that integrates multi-scale energy-water-land resource optimization with distributed hydrological modeling. The new approach provides insights into the vulnerability of water, energy and land resources to future socioeconomic and climatic change and how multi-sectoral policies, technological solutions and investments can improve the resilience and sustainability of transformation pathways while avoiding counterproductive interactions among sectors. NEST can be applied at different spatial and temporal resolutions, and is designed specifically to tap into the growing body of open access geospatial data available through national inventories and the earth system modeling community. A case study analysis of the Indus River Basin in South Asia demonstrates the capability of the model to capture important interlinkages across system transformation pathways towards the United Nations' Sustainable Development Goals, including the intersections between local and regional transboundary policies and incremental investment costs from rapidly increasing regional consumption projected over the coming decades

A [4Fe-4S]-Fe(CO)(CN)-L-cysteine intermediate is the first organometallic precursor in [FeFe] hydrogenase H-cluster bioassembly.

Biosynthesis of the [FeFe] hydrogenase active site (the 'H-cluster') requires the interplay of multiple proteins and small molecules. Among them, the radical S-adenosylmethionine enzyme HydG, a tyrosine lyase, has been proposed to generate a complex that contains an Fe(CO)2(CN) moiety that is eventually incorporated into the H-cluster. Here we describe the characterization of an intermediate in the HydG reaction: a [4Fe-4S][(Cys)Fe(CO)(CN)] species, 'Complex A', in which a CO, a CN- and a cysteine (Cys) molecule bind to the unique 'dangler' Fe site of the auxiliary [5Fe-4S] cluster of HydG. The identification of this intermediate-the first organometallic precursor to the H-cluster-validates the previously hypothesized HydG reaction cycle and provides a basis for elucidating the biosynthetic origin of other moieties of the H-cluster

Predicting cell types and genetic variations contributing to disease by combining GWAS and epigenetic data

Genome-wide association studies (GWASs) identify single nucleotide polymorphisms (SNPs) that are enriched in individuals suffering from a given disease. Most disease-associated SNPs fall into non-coding regions, so that it is not straightforward to infer phenotype or function; moreover, many SNPs are in tight genetic linkage, so that a SNP identified as associated with a particular disease may not itself be causal, but rather signify the presence of a linked SNP that is functionally relevant to disease pathogenesis. Here, we present an analysis method that takes advantage of the recent rapid accumulation of epigenomics data to address these problems for some SNPs. Using asthma as a prototypic example; we show that non-coding disease-associated SNPs are enriched in genomic regions that function as regulators of transcription, such as enhancers and promoters. Identifying enhancers based on the presence of the histone modification marks such as H3K4me1 in different cell types, we show that the location of enhancers is highly cell-type specific. We use these findings to predict which SNPs are likely to be directly contributing to disease based on their presence in regulatory regions, and in which cell types their effect is expected to be detectable. Moreover, we can also predict which cell types contribute to a disease based on overlap of the disease-associated SNPs with the locations of enhancers present in a given cell type. Finally, we suggest that it will be possible to re-analyze GWAS studies with much higher power by limiting the SNPs considered to those in coding or regulatory regions of cell types relevant to a given disease

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

Identification of elderly fallers by muscle strength measures

For efficient prevention of falls among older adults, individuals at a high risk of falling need to be identified. In this study, we searched for muscle strength measures that best identified those individuals who would fall after a gait perturbation and those who recovered their balance. Seventeen healthy older adults performed a range of muscle strength tests. We measured maximum and rate of development of ankle plantar flexion moment, knee extension moment and whole leg push-off force, as well as maximum jump height and hand grip strength. Subsequently, their capacity to regain balance after tripping over an obstacle was determined experimentally. Seven of the participants were classified as fallers based on the tripping outcome. Maximum isometric push-off force in a leg press apparatus was the best measure to identify the fallers, as cross-validation of a discriminant model with this variable resulted in the best classification (86% sensitivity and 90% specificity). Jump height and hand grip strength were strongly correlated to leg press force (r = 0.82 and 0.59, respectively) and can also be used to identify fallers, although with slightly lower specificity. These results indicate that whole leg extension strength is associated with the ability to prevent a fall after a gait perturbation and might be used to identify the elderly at risk of falling

Coordinated optimization of visual cortical maps (I) Symmetry-based analysis

In the primary visual cortex of primates and carnivores, functional architecture can be characterized by maps of various stimulus features such as orientation preference (OP), ocular dominance (OD), and spatial frequency. It is a long-standing question in theoretical neuroscience whether the observed maps should be interpreted as optima of a specific energy functional that summarizes the design principles of cortical functional architecture. A rigorous evaluation of this optimization hypothesis is particularly demanded by recent evidence that the functional architecture of OP columns precisely follows species invariant quantitative laws. Because it would be desirable to infer the form of such an optimization principle from the biological data, the optimization approach to explain cortical functional architecture raises the following questions: i) What are the genuine ground states of candidate energy functionals and how can they be calculated with precision and rigor? ii) How do differences in candidate optimization principles impact on the predicted map structure and conversely what can be learned about an hypothetical underlying optimization principle from observations on map structure? iii) Is there a way to analyze the coordinated organization of cortical maps predicted by optimization principles in general? To answer these questions we developed a general dynamical systems approach to the combined optimization of visual cortical maps of OP and another scalar feature such as OD or spatial frequency preference.Comment: 90 pages, 16 figure

Cognitive performance in relapsing remitting multiple sclerosis: A longitudinal study in daily practice using a brief computerized cognitive battery

<p>Abstract</p> <p>Background</p> <p>There is need for a cognitive test battery that can be easily used in clinical practice to detect or monitor cognitive performance in patients with multiple sclerosis (MS). In order to conduct, in this patient group, a preliminary investigation of the validity and utility of a brief computerized battery, the Cognitive Drug Research (CDR) battery, we longitudinally assessed cognition in patients with relapsing remitting (RR) MS.</p> <p>Methods</p> <p>Forty-three mildly disabled, clinically active RRMS patients were repeatedly assessed with the Digit Symbol Substitution Test (DSST), Paced Auditory Serial Addition Test (PASAT) and five composite scores derived from the CDR computerized cognitive test system (CDR System): Power of Attention, Continuity of Attention, Quality of Working Memory, Quality of Episodic Memory and Speed of Memory. The Multiple Sclerosis Functional Composite (MSFC) and Expanded Disability Status Scale (EDSS) measured disability.</p> <p>Results</p> <p>The composite scores from the CDR battery generally showed excellent test-retest reliability over the repeated assessments, though was low on occasions for the Quality of Working Memory and Quality of Episodic Memory measures. The CDR measures tended to be highly correlated with other measures of cognition (DSST and PASAT) and were also strongly related to disability (EDSS and MSFC). Baseline scores indicated large impairments to visual information processing speed and attention (DSST, Cohen's d 1.1; Power of Attention d 1.4 [reaction time on tasks of focussed and sustained attention]), and a moderate impairment both to sustained attention (Continuity of Attention d 0.6) and complex information processing speed (Speed of memory d 0.7 [reaction time on tasks of working and episodic Memory]), when compared to normative data derived from healthy volunteers enrolled in a series of separate, prior clinical trials. Working memory (Quality of Working Memory) and episodic memory (Quality of Episodic Memory) were unimpaired.</p> <p>Conclusions</p> <p>Preliminary validation of the CDR System indicated that for most, but not all measures psychometric properties were adequate and the measures were related to disability (EDSS and MSFC) and other measures of cognition.</p