Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

Molecular dynamics simulations of oscillatory Couette flows with slip boundary conditions

The effect of interfacial slip on steady-state and time-periodic flows of monatomic liquids is investigated using non-equilibrium molecular dynamics simulations. The fluid phase is confined between atomically smooth rigid walls, and the fluid flows are induced by moving one of the walls. In steady shear flows, the slip length increases almost linearly with shear rate. We found that the velocity profiles in oscillatory flows are well described by the Stokes flow solution with the slip length that depends on the local shear rate. Interestingly, the rate dependence of the slip length obtained in steady shear flows is recovered when the slip length in oscillatory flows is plotted as a function of the local shear rate magnitude. For both types of flows, the friction coefficient at the liquid-solid interface correlates well with the structure of the first fluid layer near the solid wall.Comment: 31 pages, 11 figure

An integrated biostratigraphy and seismic stratigraphy for the late Neogene continental margin succession in northern Taranaki Basin, New Zealand

Our aim has been to develop an integrated biostratigraphy and seismic stratigraphy for the Pliocene and Pleistocene formations (Ariki, Mangaa, Giant Foresets) in northern Taranaki Basin to better understand the evolution of the modern continental margin offshore central-western North Island, New Zealand. Detailed mapping of seismic reflectors in part of the basin, when compared with correlations of late Neogene stage boundaries between 11 well sections, has highlighted crossover between the datasets. To help resolve this issue, the biostratigraphy of the Pliocene-Pleistocene parts of each of four well sections (Arawa-1, Ariki-1, Kora-1, and Wainui-1) has been re-examined using a dense suite of samples. In addition, the biostratigraphy of seven other well sections (Awatea-1, Kahawai-1, Mangaa-1, Taimana-1, Tangaroa-1, Te Kumi-1, and Turi-1) has been re-evaluated. The crossover is partly attributed to a combination of sampling resolution inherent in exploration well sections, the mixed nature of cuttings samples, and the general scarcity of age-diagnostic planktic foraminifera in the late Neogene formations. The achievement of seismic closure suggests that error in the mapping of the seismic reflectors is not a significant source of the uncertainty (crossover). We have developed a workable time-stratigraphic framework by qualitatively weighting the biostratigraphic data in each of the well sections, thereby identifying the parts of particular well sections with the highest resolution microfossil data and the optimal stratigraphic position of stage boundaries with respect to the mapped seismic horizons/seismic units. Hence, it is possible to assign the known numerical ages for these stage boundaries to reflection horizons/seismic units mapped within the basin. We have applied this information to produce a series of isopach maps for successive stage boundaries that help show the sedimentary evolution of the continental margin succession west of central North Island

Improved model identification for non-linear systems using a random subsampling and multifold modelling (RSMM) approach

In non-linear system identification, the available observed data are conventionally partitioned into two parts: the training data that are used for model identification and the test data that are used for model performance testing. This sort of 'hold-out' or 'split-sample' data partitioning method is convenient and the associated model identification procedure is in general easy to implement. The resultant model obtained from such a once-partitioned single training dataset, however, may occasionally lack robustness and generalisation to represent future unseen data, because the performance of the identified model may be highly dependent on how the data partition is made. To overcome the drawback of the hold-out data partitioning method, this study presents a new random subsampling and multifold modelling (RSMM) approach to produce less biased or preferably unbiased models. The basic idea and the associated procedure are as follows. First, generate K training datasets (and also K validation datasets), using a K-fold random subsampling method. Secondly, detect significant model terms and identify a common model structure that fits all the K datasets using a new proposed common model selection approach, called the multiple orthogonal search algorithm. Finally, estimate and refine the model parameters for the identified common-structured model using a multifold parameter estimation method. The proposed method can produce robust models with better generalisation performance

The UK market for energy service contracts in 2014–2015

This paper provides an overview of the UK market for energy service contracts in 2014 and highlights the growing role of intermediaries. Using information from secondary literature and interviews, it identifies the businesses offering energy service contracts, the sectors and organisations that are purchasing those contracts, the types of contract that are available, the areas of market growth and the reasons for that growth. The paper finds that the UK market is relatively large, highly diverse, concentrated in particular sectors and types of site and overwhelmingly focused upon established technologies with high rates of return. A major driver is the emergence of procurement frameworks for energy service contracts in the public sector. These act as intermediaries between clients and contractors, thereby lowering transaction costs and facilitating learning. The market is struggling to become established in commercial offices, largely as a result of split incentives, and is unlikely to develop further in this sector without different business models, tenancy arrangements and policy initiatives. Overall, the paper concludes that energy service contracts can play an important role in the transition to a low-carbon economy, especially when supported by intermediaries, but their potential is still limited by high transaction costs

Interplay of buried histidine protonation and protein stability in prion misfolding

Misofolding of mammalian prion proteins (PrP) is believed to be the cause of a group of rare and fatal neurodegenerative diseases. Despite intense scrutiny however, the mechanism of the misfolding reaction remains unclear. We perform nuclear Magnetic Resonance and thermodynamic stability measurements on the C-terminal domains (residues 90–231) of two PrP variants exhibiting different pH-induced susceptibilities to aggregation: the susceptible hamster prion (GHaPrP) and its less susceptible rabbit homolog (RaPrP). The pKa of histidines in these domains are determined from titration experiments, and proton-exchange rates are measured at pH 5 and pH 7. A single buried highly conserved histidine, H187/H186 in GHaPrP/RaPrP, exhibited a markedly down shifted pKa ~5 for both proteins. However, noticeably larger pH-induced shifts in exchange rates occur for GHaPrP versus RaPrP. Analysis of the data indicates that protonation of the buried histidine destabilizes both PrP variants, but produces a more drastic effect in the less stable GHaPrP. This interpretation is supported by urea denaturation experiments performed on both PrP variants at neutral and low pH, and correlates with the difference in disease susceptibility of the two species, as expected from the documented linkage between destabilization of the folded state and formation of misfolded and aggregated species

Heavy metal pollution and co-selection for antibiotic resistance: A microbial palaeontology approach

This is the final version. Available on open access from Elsevier via the DOI in this recordFrequent and persistent heavy metal pollution has profound effects on the composition and activity of microbial communities. Heavy metals select for metal resistance but can also co-select for resistance to antibiotics, which is a global health concern. We here document metal concentration, metal resistance and antibiotic resistance along a sediment archive from a pond in the North West of the United Kingdom covering over a century of anthropogenic pollution. We specifically focus on zinc, as it is a ubiquitous and toxic metal contaminant known to co-select for antibiotic resistance, to assess the impact of temporal variation in heavy metal pollution on microbial community diversity and to quantify the selection effects of differential heavy metal exposure on antibiotic resistance. Zinc concentration and bioavailability was found to vary over the core, likely reflecting increased industrialisation around the middle of the 20th century. Zinc concentration had a significant effect on bacterial community composition, as revealed by a positive correlation between the level of zinc tolerance in culturable bacteria and zinc concentration. The proportion of zinc resistant isolates was also positively correlated with resistance to three clinically relevant antibiotics (oxacillin, cefotaxime and trimethoprim). The abundance of the class 1 integron-integrase gene, intI1, marker for anthropogenic pollutants correlated with the prevalence of zinc- and cefotaxime resistance but not with oxacillin and trimethoprim resistance. Our microbial palaeontology approach reveals that metal-contaminated sediments from depths that pre-date the use of antibiotics were enriched in antibiotic resistant bacteria, demonstrating the pervasive effects of metal-antibiotic co-selection in the environment.University of Exete

Olfactory and solitary chemosensory cells: two different chemosensory systems in the nasal cavity of the American alligator, Alligator mississippiensis

<p>Abstract</p> <p>Background</p> <p>The nasal cavity of all vertebrates houses multiple chemosensors, either innervated by the Ist (olfactory) or the Vth (trigeminal) cranial nerve. Various types of receptor cells are present, either segregated in different compartments (e.g. in rodents) or mingled in one epithelium (e.g. fish). In addition, solitary chemosensory cells have been reported for several species. Alligators which seek their prey both above and under water have only one nasal compartment. Information about their olfactory epithelium is limited. Since alligators seem to detect both volatile and water-soluble odour cues, I tested whether different sensory cell types are present in the olfactory epithelium.</p> <p>Results</p> <p>Electron microscopy and immunocytochemistry were used to examine the sensory epithelium of the nasal cavity of the American alligator. Almost the entire nasal cavity is lined with olfactory (sensory) epithelium. Two types of olfactory sensory neurons are present. Both types bear cilia as well as microvilli at their apical endings and express the typical markers for olfactory neurons. The density of these olfactory neurons varies along the nasal cavity. In addition, solitary chemosensory cells innervated by trigeminal nerve fibres, are intermingled with olfactory sensory neurons. Solitary chemosensory cells express components of the PLC-transduction cascade found in solitary chemosensory cells in rodents.</p> <p>Conclusion</p> <p>The nasal cavity of the American alligator contains two different chemosensory systems incorporated in the same sensory epithelium: the olfactory system proper and solitary chemosensory cells. The olfactory system contains two morphological distinct types of ciliated olfactory receptor neurons.</p