Extreme State Aggregation Beyond MDPs

We consider a Reinforcement Learning setup where an agent interacts with an environment in observation-reward-action cycles without any (esp.\ MDP) assumptions on the environment. State aggregation and more generally feature reinforcement learning is concerned with mapping histories/raw-states to reduced/aggregated states. The idea behind both is that the resulting reduced process (approximately) forms a small stationary finite-state MDP, which can then be efficiently solved or learnt. We considerably generalize existing aggregation results by showing that even if the reduced process is not an MDP, the (q-)value functions and (optimal) policies of an associated MDP with same state-space size solve the original problem, as long as the solution can approximately be represented as a function of the reduced states. This implies an upper bound on the required state space size that holds uniformly for all RL problems. It may also explain why RL algorithms designed for MDPs sometimes perform well beyond MDPs.Comment: 28 LaTeX pages. 8 Theorem

NeAT: a toolbox for the analysis of biological networks, clusters, classes and pathways

The network analysis tools (NeAT) (http://rsat.ulb.ac.be/neat/) provide a user-friendly web access to a collection of modular tools for the analysis of networks (graphs) and clusters (e.g. microarray clusters, functional classes, etc.). A first set of tools supports basic operations on graphs (comparison between two graphs, neighborhood of a set of input nodes, path finding and graph randomization). Another set of programs makes the connection between networks and clusters (graph-based clustering, cliques discovery and mapping of clusters onto a network). The toolbox also includes programs for detecting significant intersections between clusters/classes (e.g. clusters of co-expression versus functional classes of genes). NeAT are designed to cope with large datasets and provide a flexible toolbox for analyzing biological networks stored in various databases (protein interactions, regulation and metabolism) or obtained from high-throughput experiments (two-hybrid, mass-spectrometry and microarrays). The web interface interconnects the programs in predefined analysis flows, enabling to address a series of questions about networks of interest. Each tool can also be used separately by entering custom data for a specific analysis. NeAT can also be used as web services (SOAP/WSDL interface), in order to design programmatic workflows and integrate them with other available resources

NeAT: a toolbox for the analysis of biological networks, clusters, classes and pathways

The network analysis tools (NeAT) (http://rsat.ulb.ac.be/neat/) provide a user-friendly web access to a collection of modular tools for the analysis of networks (graphs) and clusters (e.g. microarray clusters, functional classes, etc.). A first set of tools supports basic operations on graphs (comparison between two graphs, neighborhood of a set of input nodes, path finding and graph randomization). Another set of programs makes the connection between networks and clusters (graph-based clustering, cliques discovery and mapping of clusters onto a network). The toolbox also includes programs for detecting significant intersections between clusters/classes (e.g. clusters of co-expression versus functional classes of genes). NeAT are designed to cope with large datasets and provide a flexible toolbox for analyzing biological networks stored in various databases (protein interactions, regulation and metabolism) or obtained from high-throughput experiments (two-hybrid, mass-spectrometry and microarrays). The web interface interconnects the programs in predefined analysis flows, enabling to address a series of questions about networks of interest. Each tool can also be used separately by entering custom data for a specific analysis. NeAT can also be used as web services (SOAP/WSDL interface), in order to design programmatic workflows and integrate them with other available resources

How do risk attitudes affect measured confidence?

We examine the relationship between confidence in own absolute performance and risk attitudes using two confidence elicitation procedures: self-reported (non-incentivised) confidence and an incentivised procedure that elicits the certainty equivalent of a bet based on performance. The former procedure reproduces the “hard-easy effect” (underconfidence in easy tasks and overconfidence in hard tasks) found in a large number of studies using non-incentivised self-reports. The latter procedure produces general underconfidence, which is significantly reduced, but not eliminated when we filter out the effects of risk attitudes. Finally, we find that self-reported confidence correlates significantly with features of individual risk attitudes including parameters of individual probability weighting

Radiocarbon dating and cultural dynamics across Mongolia’s early pastoral transition

All necessary permits were obtained for the described study, which complied with all relevant regulations. Collaboration contract between the Max Planck Institute for the Science of Human HIstory and the National University of Mongolia began on the 10th November, 2016. Export number 10/413 (7b/52) was received on the 2nd Feb, 2017 (#A0109258, MN DE 7 643). This research was supported by the Max Planck Institute for the Science of Human History. Special thanks to Dr. Katerina Douka and the Oxford Radiocarbon Accelerator Laboratory for conducting 14C analysis, and to all of the original excavators and authors who published the radiocarbon dates cited in this study.Peer reviewedPublisher PD

Sustainability and the capability approach: from theory to practice?

YesThe capability approach and sustainability can be connected in numerous ways. One could think of sustainability as a self-contained domain of human analysis – thus there could be theories of sustainability and there may be difficulties in this domain as elsewhere in moving from theory to policy or practice. Thus, capability approach could be considered as an additional lens that can facilitate the transition from sustainability theory to practice; alternatively one could think of the capability approach as offering an alternative paradigm and thus build on both theories and then find ways to move from theory to practice. In this chapter, both of these approaches are recognised and discussed. The capability approach is mainly about enhancing substantive freedoms- we examine the conjectures whether an approach of increasing freedoms is compatible with sustainability and whether freedoms are sufficient for sustainability. We use the case of Mongolia to explore some of these issues of application

Pastoralism and Emergent Complex Settlement in the Middle Bronze Age, Azerbaijan: Isotopic analyses of mobility strategies in transformation

Objectives. This article explores the scale and seasonal patterns of mobility at the complex settlement site of Qızqala during the Middle Bronze Age (2400–1,500 BCE). By integrating human bone, teeth, and environmental samples this research tests the hypothesis of the persistent importance of community-wide seasonal pastoral transhumance during the early formation of complex settlement systems of the South Caucasus. Methods. This research applies stable oxygen and radiogenic strontium isotope analyses on incremental samples of human tooth enamel, bulk tooth enamel, and bone to resolve mobility patterns. Sequential and bulk sampling techniques elucidate seasonal and residential mobility behaviors. Extensive environmental isotope samples of plant and water were collected through regional survey and establish local and regional isotopic baselines, which are compared to human isotope analysis results. Results. Qızqala individuals exhibit low isotopic variability compared to regional contemporaries. 87Sr/86Sr ratios from human remains indicate seasonal and residential isotopic variability within the baseline ranges of local landscapes. δ18O values display erratic patterns, but correspond to seasonal variability with fluctuations between highland and lowland altitudinal zone baseline values. Conclusions. Results suggest that isotopic analysis of multiple elements and sequential enamel samples offers finer resolution on the complexities of human mobility strategies and elucidate the daily lives of often overlooked mobile populations. Higher resolution of individual mobility reveals shared routine behaviors that underscore the importance of diverse social collaborations in forming complex polities in the South Caucasus

Selective Serotonin Reuptake Inhibitor (SSRI) Antidepressants in Pregnancy and Congenital Anomalies: Analysis of Linked Databases in Wales, Norway and Funen, Denmark

Background: Hypothesised associations between in utero exposure to selective serotonin reuptake inhibitors (SSRIs) and congenital anomalies, particularly congenital heart defects (CHD), remain controversial. We investigated the putative teratogenicity of SSRI prescription in the 91 days either side of first day of last menstrual period (LMP). Methods and Findings: Three population-based EUROCAT congenital anomaly registries- Norway (2004–2010), Wales (2000–2010) and Funen, Denmark (2000–2010)—were linked to the electronic healthcare databases holding prospectively collected prescription information for all pregnancies in the timeframes available. We included 519,117 deliveries, including foetuses terminated for congenital anomalies, with data covering pregnancy and the preceding quarter, including 462,641 with data covering pregnancy and one year either side. For SSRI exposures 91 days either side of LMP, separately and together, odds ratios with 95% confidence intervals (ORs, 95%CI) for all major anomalies were estimated. We also explored: pausing or discontinuing SSRIs preconception, confounding, high dose regimens, and, in Wales, diagnosis of depression. Results were combined in meta-analyses. SSRI prescription 91 days either side of LMP was associated with increased prevalence of severe congenital heart defects (CHD) (as defined by EUROCAT guide 1.3, 2005) (34/12,962 [0.26%] vs. 865/506,155 [0.17%] OR 1.50, 1.06–2.11), and the composite adverse outcome of 'anomaly or stillbirth' (473/12962, 3.65% vs. 15829/506,155, 3.13%, OR 1.13, 1.03–1.24). The increased prevalence of all major anomalies combined did not reach statistical significance (3.09% [400/12,962] vs. 2.67% [13,536/506,155] OR 1.09, 0.99–1.21). Adjusting for socio-economic status left ORs largely unchanged. The prevalence of anomalies and severe CHD was reduced when SSRI prescriptions were stopped or paused preconception, and increased when >1 prescription was recorded, but differences were not statistically significant. The dose-response relationship between severe CHD and SSRI dose (meta-regression OR 1.49, 1.12–1.97) was consistent with SSRI-exposure related risk. Analyses in Wales suggested no associations between anomalies and diagnosed depression. Conclusion: The additional absolute risk of teratogenesis associated with SSRIs, if causal, is small. However, the high prevalence of SSRI use augments its public health importance, justifying modifications to preconception care

The Princeton Protein Orthology Database (P-POD): A Comparative Genomics Analysis Tool for Biologists

Many biological databases that provide comparative genomics information and tools are now available on the internet. While certainly quite useful, to our knowledge none of the existing databases combine results from multiple comparative genomics methods with manually curated information from the literature. Here we describe the Princeton Protein Orthology Database (P-POD, http://ortholog.princeton.edu), a user-friendly database system that allows users to find and visualize the phylogenetic relationships among predicted orthologs (based on the OrthoMCL method) to a query gene from any of eight eukaryotic organisms, and to see the orthologs in a wider evolutionary context (based on the Jaccard clustering method). In addition to the phylogenetic information, the database contains experimental results manually collected from the literature that can be compared to the computational analyses, as well as links to relevant human disease and gene information via the OMIM, model organism, and sequence databases. Our aim is for the P-POD resource to be extremely useful to typical experimental biologists wanting to learn more about the evolutionary context of their favorite genes. P-POD is based on the commonly used Generic Model Organism Database (GMOD) schema and can be downloaded in its entirety for installation on one's own system. Thus, bioinformaticians and software developers may also find P-POD useful because they can use the P-POD database infrastructure when developing their own comparative genomics resources and database tools

Baseline characteristics of patients in the reduction of events with darbepoetin alfa in heart failure trial (RED-HF)

<p>Aims: This report describes the baseline characteristics of patients in the Reduction of Events with Darbepoetin alfa in Heart Failure trial (RED-HF) which is testing the hypothesis that anaemia correction with darbepoetin alfa will reduce the composite endpoint of death from any cause or hospital admission for worsening heart failure, and improve other outcomes.</p> <p>Methods and results: Key demographic, clinical, and laboratory findings, along with baseline treatment, are reported and compared with those of patients in other recent clinical trials in heart failure. Compared with other recent trials, RED-HF enrolled more elderly [mean age 70 (SD 11.4) years], female (41%), and black (9%) patients. RED-HF patients more often had diabetes (46%) and renal impairment (72% had an estimated glomerular filtration rate <60 mL/min/1.73 m2). Patients in RED-HF had heart failure of longer duration [5.3 (5.4) years], worse NYHA class (35% II, 63% III, and 2% IV), and more signs of congestion. Mean EF was 30% (6.8%). RED-HF patients were well treated at randomization, and pharmacological therapy at baseline was broadly similar to that of other recent trials, taking account of study-specific inclusion/exclusion criteria. Median (interquartile range) haemoglobin at baseline was 112 (106–117) g/L.</p> <p>Conclusion: The anaemic patients enrolled in RED-HF were older, moderately to markedly symptomatic, and had extensive co-morbidity.</p&gt