Data-efficient reinforcement learning in continuous state-action Gaussian-POMDPs

We present a data-efficient reinforcement learning method for continuous state-action systems under significant observation noise. Data-efficient solutions under small noise exist, such as PILCO which learns the cartpole swing-up task in 30s. PILCO evaluates policies by planning state-trajectories using a dynamics model. However, PILCO applies policies to the observed state, therefore planning in observation space. We extend PILCO with filtering to instead plan in belief space, consistent with partially observable Markov decisions process (POMDP) planning. This enables data-efficient learning under significant observation noise, outperforming more naive methods such as post-hoc application of a filter to policies optimised by the original (unfiltered) PILCO algorithm. We test our method on the cartpole swing-up task, which involves nonlinear dynamics and requires nonlinear control

Aspects of Discrete Breathers and New Directions

We describe results concerning the existence proofs of Discrete Breathers (DBs) in the two classes of dynamical systems with optical linear phonons and with acoustic linear phonons. A standard approach is by continuation of DBs from an anticontinuous limit. A new approach, which is purely variational, is presented. We also review some numerical results on intraband DBs in random nonlinear systems. Some non-conventional physical applications of DBs are suggested. One of them is understanding slow relaxation properties of glassy materials. Another one concerns energy focusing and transport in biomolecules by targeted energy transfer of DBs. A similar theory could be used for describing targeted charge transfer of nonlinear electrons (polarons) and, more generally, for targeted transfer of several excitations (e.g. Davydov soliton).Comment: to appear in the Proceedings of NATO Advanced Research Workshop "Nonlinearity and Disorder: Theory and Applications", Tashkent,Uzbekistan,October 1-6, 200

PILER-CR: Fast and accurate identification of CRISPR repeats

BACKGROUND: Sequencing of prokaryotic genomes has recently revealed the presence of CRISPR elements: short, highly conserved repeats separated by unique sequences of similar length. The distinctive sequence signature of CRISPR repeats can be found using general-purpose repeat- or pattern-finding software tools. However, the output of such tools is not always ideal for studying these repeats, and significant effort is sometimes needed to build additional tools and perform manual analysis of the output. RESULTS: We present PILER-CR, a program specifically designed for the identification and analysis of CRISPR repeats. The program executes rapidly, completing a 5 Mb genome in around 5 seconds on a current desktop computer. We validate the algorithm by manual curation and by comparison with published surveys of these repeats, finding that PILER-CR has both high sensitivity and high specificity. We also present a catalogue of putative CRISPR repeats identified in a comprehensive analysis of 346 prokaryotic genomes. CONCLUSION: PILER-CR is a useful tool for rapid identification and classification of CRISPR repeats. The software is donated to the public domain. Source code and a Linux binary are freely available at

Rapid global ocean-atmosphere response to Southern Ocean freshening during the last glacial.

Contrasting Greenland and Antarctic temperatures during the last glacial period (115,000 to 11,650 years ago) are thought to have been driven by imbalances in the rates of formation of North Atlantic and Antarctic Deep Water (the 'bipolar seesaw'). Here we exploit a bidecadally resolved 14C data set obtained from New Zealand kauri (Agathis australis) to undertake high-precision alignment of key climate data sets spanning iceberg-rafted debris event Heinrich 3 and Greenland Interstadial (GI) 5.1 in the North Atlantic (~30,400 to 28,400 years ago). We observe no divergence between the kauri and Atlantic marine sediment 14C data sets, implying limited changes in deep water formation. However, a Southern Ocean (Atlantic-sector) iceberg rafted debris event appears to have occurred synchronously with GI-5.1 warming and decreased precipitation over the western equatorial Pacific and Atlantic. An ensemble of transient meltwater simulations shows that Antarctic-sourced salinity anomalies can generate climate changes that are propagated globally via an atmospheric Rossby wave train.A challenge for testing mechanisms of past climate change is the precise correlation of palaeoclimate records. Here, through climate modelling and the alignment of terrestrial, ice and marine 14C and 10Be records, the authors show that Southern Ocean freshwater hosing can trigger global change

Genome sequencing of the extinct Eurasian wild aurochs, Bos primigenius, illuminates the phylogeography and evolution of cattle

Background Domestication of the now-extinct wild aurochs, Bos primigenius, gave rise to the two major domestic extant cattle taxa, B. taurus and B. indicus. While previous genetic studies have shed some light on the evolutionary relationships between European aurochs and modern cattle, important questions remain unanswered, including the phylogenetic status of aurochs, whether gene flow from aurochs into early domestic populations occurred, and which genomic regions were subject to selection processes during and after domestication. Here, we address these questions using whole-genome sequencing data generated from an approximately 6,750-year-old British aurochs bone and genome sequence data from 81 additional cattle plus genome-wide single nucleotide polymorphism data from a diverse panel of 1,225 modern animals. Results Phylogenomic analyses place the aurochs as a distinct outgroup to the domestic B. taurus lineage, supporting the predominant Near Eastern origin of European cattle. Conversely, traditional British and Irish breeds share more genetic variants with this aurochs specimen than other European populations, supporting localized gene flow from aurochs into the ancestors of modern British and Irish cattle, perhaps through purposeful restocking by early herders in Britain. Finally, the functions of genes showing evidence for positive selection in B. taurus are enriched for neurobiology, growth, metabolism and immunobiology, suggesting that these biological processes have been important in the domestication of cattle. Conclusions This work provides important new information regarding the origins and functional evolution of modern cattle, revealing that the interface between early European domestic populations and wild aurochs was significantly more complex than previously thought

Uteroplacental bleeding disorders during pregnancy: do missing paternal characteristics influence risk?

BACKGROUND: Several studies have assessed the risks of uteroplacental bleeding disorders in relation to maternal characteristics. The association between uteroplacental bleeding disorders and paternal characteristics, however, has received considerably less attention. Data on paternal demographics, notably race and age, from birth certificate data are becoming increasingly incomplete in recent years. This pattern of increasingly underreporting of paternal demographic data led us to speculate that pregnancies for which paternal characteristics are partially or completely missing may be associated with increased risk for uteroplacental bleeding disorders. The objective of this study is to examine the association between placenta previa and placental abruption and missing paternal age and race. METHODS: A retrospective cohort study using U.S. linked birth/infant death data from 1995 through 2001 (n = 26,336,549) was performed. Risks of placenta previa and placental abruption among: (i) pregnancies with complete paternal age and race data; (ii) paternal age only missing; (iii) paternal race only missing; and (iv) both paternal age and race missing, were evaluated. Relative risk (RR) with 95% confidence interval (CI) for placenta previa and placental abruption by missing paternal characteristics were derived after adjusting for confounders. RESULTS: Adjusted RR for placental abruption were 1.30 (95% CI 1.24, 1.37), 1.00 (95% CI 0.95, 1.05), and 1.08 (95% CI 1.06, 1.10) among pregnancies with "paternal age only", "paternal race only", and "both paternal age and race" missing, respectively. The increased risk of placental abruption among the "paternal age only missing" category is partly explained by increased risks among whites aged 20–29 years, and among blacks aged ≥30 years. However, no clear patterns in the associations between missing paternal characteristics and placenta previa were evident. CONCLUSION: Missing paternal characteristics are associated with increased risk of placental abruption, likely mediated through low socio-economic conditions

Optical biosensor differentiates signaling of endogenous PAR1 and PAR2 in A431 cells

<p>Abstract</p> <p>Background</p> <p>Protease activated receptors (PARs) consist of a family of four G protein-coupled receptors. Many types of cells express several PARs, whose physiological significance is mostly unknown.</p> <p>Results</p> <p>Here, we show that non-invasive resonant waveguide grating (RWG) biosensor differentiates signaling of endogenous protease activated receptor subtype 1 (PAR₁) and 2 (PAR₂) in human epidermoid carcinoma A431 cells. The biosensor directly measures dynamic mass redistribution (DMR) resulted from ligand-induced receptor activation in adherent cells. In A431, both PAR₁and PAR₂agonists, but neither PAR₃nor PAR₄agonists, trigger dose-dependent Ca²⁺mobilization as well as G_q-type DMR signals. Both Ca²⁺flux and DMR signals display comparable desensitization patterns upon repeated stimulation with different combinations of agonists. However, PAR₁and PAR₂exhibit distinct kinetics of receptor re-sensitization. Furthermore, both trypsin- and thrombin-induced Ca²⁺flux signals show almost identical dependence on cell surface cholesterol level, but their corresponding DMR signals present different sensitivities.</p> <p>Conclusion</p> <p>Optical biosensor provides an alternative readout for examining receptor activation under physiologically relevant conditions, and differentiates the signaling of endogenous PAR₁and PAR₂in A431.</p

Sea-ice dynamics in an Arctic coastal polynya during the past 6500 years

The production of high-salinity brines during sea-ice freezing in circum-arctic coastal polynyas is thought to be part of northern deep water formation as it supplies additional dense waters to the Atlantic meridional overturning circulation system. To better predict the effect of possible future summer ice-free conditions in the Arctic Ocean on global climate, it is important to improve our understanding of how climate change has affected sea-ice and brine formation, and thus finally dense water formation during the past. Here, we show temporal coherence between sea-ice conditions in a key Arctic polynya (Storfjorden, Svalbard) and patterns of deep water convection in the neighbouring Nordic Seas over the last 6500 years. A period of frequent sea-ice melting and freezing between 6.5 and 2.8 ka BP coincided with enhanced deep water renewal in the Nordic Seas. Near-permanent sea-ice cover and low brine rejection after 2.8 ka BP likely reduced the overflow of high-salinity shelf waters, concomitant with a gradual slow down of deep water convection in the Nordic Seas, which occurred along with a regional expansion in sea-ice and surface water freshening. The Storfjorden polynya sea-ice factory restarted at ~0.5 ka BP, coincident with renewed deep water penetration to the Arctic and climate amelioration over Svalbard. The identified synergy between Arctic polynya sea-ice conditions and deep water convection during the present interglacial is an indication of the potential consequences for ocean ventilation during states with permanent sea-ice cover or future Arctic ice-free conditions

Optimization Formulations for the Design of Low Embodied Energy Structures Made from Reused Elements

The building sector is one of the major contributors to material resource consumption, greenhouse gas emission and waste production. Load-bearing systems have a particularly large environmental impact because of their material and energy intensive manufacturing process. This paper aims to address the reduction of building structures environmental impacts through reusing structural elements for multiple service lives. Reuse avoids sourcing raw materials and requires little energy for reprocessing. However, to design a new structure reusing elements available from a stock is a challenging problem of combinatorial nature. This is because the structural system layout is a result of the available elements’ mechanical and geometric properties. In this paper, structural optimization formulations are proposed to design truss systems from available stock elements. Minimization of weight, cut-off waste and embodied energy are the objective functions subject to ultimate and serviceability constraints. Case studies focusing on embodied energy minimization are presented for: (1) three roof systems with predefined geometry and topology; (2) a bridge structure whose topology is optimized using the ground structure approach; (3) a geometry optimization to better match the optimal topology from 2 and available stock element lengths. In order to benchmark the energy savings through reuse, the optimal layouts obtained with the proposed methods are compared to weight-optimized solutions made of new material. For these case studies, the methods proposed in this work enable reusing stock elements to design structures embodying up to 71% less energy and hence having a significantly lower environmental impact with respect to structures made of new material

The site of embolization related to infarct size, oedema and clinical outcome in a rat stroke model - further translational stroke research

<p>Abstract</p> <p>Background and purpose</p> <p>Reliable models are essential for translational stroke research to study the pathophysiology of ischaemic stroke in an effort to find therapies that may ultimately reduce oedema, infarction and mortality in the clinic. The purpose of this study was to investigate the relation between the site of arterial embolization and the subsequent oedema, infarction and clinical outcome in a rat embolic stroke model.</p> <p>Methods</p> <p>Thirty-six male Sprague-Dawley rats were thromboembolized into the internal carotid artery. The site of occlusion was demonstrated by arteriography. Following histological preparation and evaluation, the size of the hemispheres and the infarcts were measured by quantitative histology and planimetry. Another parallel stroke model study was subsequently examined to investigate if the conclusions from the first study could be applied to the second study.</p> <p>Results</p> <p>The median size of the infarct was 40% of the ipsilateral hemisphere in both the 19 animals with occlusion localised to the intracranial part of the internal carotid artery and in the 11 animals where the main trunk of the middle cerebral artery was occluded. In 5 animals, occlusion of the extracranial part of the internal carotid artery resulted in significantly smaller infarcts compared to other groups (p < 0.01). Another independent study re-confirmed these results. Furthermore, significant correlations (R > 0.76, p < 0.0001) were found between 1) cortical, subcortical, and total infarct volumes, 2) oedema in percent of the left hemisphere, 3) clinical score before termination and 4) postoperative weight loss.</p> <p>Conclusions</p> <p>Distal occlusions of the intracranial part of the internal carotid or middle cerebral arteries resulted in comparable large sized infarctions and oedema. This indicates that investigators do not need a similar number of such occlusions in each experimental group. Contrary to observations in the clinic, distal internal carotid artery occlusions did not result in worse outcome than middle cerebral stem occlusions, but this finding may be explained by the controlled emboli size in this experimental stroke model.</p