Maximum Reward Formulation In Reinforcement Learning

Reinforcement learning (RL) algorithms typically deal with maximizing the expected cumulative return (discounted or undiscounted, finite or infinite horizon). However, several crucial applications in the real world, such as drug discovery, do not fit within this framework because an RL agent only needs to identify states (molecules) that achieve the highest reward within a trajectory and does not need to optimize for the expected cumulative return. In this work, we formulate an objective function to maximize the expected maximum reward along a trajectory, derive a novel functional form of the Bellman equation, introduce the corresponding Bellman operators, and provide a proof of convergence. Using this formulation, we achieve state-of-the-art results on the task of molecule generation that mimics a real-world drug discovery pipeline.Comment: 13 pages, 5 figure

A Brownian particle in a microscopic periodic potential

We study a model for a massive test particle in a microscopic periodic potential and interacting with a reservoir of light particles. In the regime considered, the fluctuations in the test particle's momentum resulting from collisions typically outweigh the shifts in momentum generated by the periodic force, and so the force is effectively a perturbative contribution. The mathematical starting point is an idealized reduced dynamics for the test particle given by a linear Boltzmann equation. In the limit that the mass ratio of a single reservoir particle to the test particle tends to zero, we show that there is convergence to the Ornstein-Uhlenbeck process under the standard normalizations for the test particle variables. Our analysis is primarily directed towards bounding the perturbative effect of the periodic potential on the particle's momentum.Comment: 60 pages. We reorganized the article and made a few simplifications of the conten

Caveolin-1 protects B6129 mice against Helicobacter pylori gastritis.

Caveolin-1 (Cav1) is a scaffold protein and pathogen receptor in the mucosa of the gastrointestinal tract. Chronic infection of gastric epithelial cells by Helicobacter pylori (H. pylori) is a major risk factor for human gastric cancer (GC) where Cav1 is frequently down-regulated. However, the function of Cav1 in H. pylori infection and pathogenesis of GC remained unknown. We show here that Cav1-deficient mice, infected for 11 months with the CagA-delivery deficient H. pylori strain SS1, developed more severe gastritis and tissue damage, including loss of parietal cells and foveolar hyperplasia, and displayed lower colonisation of the gastric mucosa than wild-type B6129 littermates. Cav1-null mice showed enhanced infiltration of macrophages and B-cells and secretion of chemokines (RANTES) but had reduced levels of CD25+ regulatory T-cells. Cav1-deficient human GC cells (AGS), infected with the CagA-delivery proficient H. pylori strain G27, were more sensitive to CagA-related cytoskeletal stress morphologies ("humming bird") compared to AGS cells stably transfected with Cav1 (AGS/Cav1). Infection of AGS/Cav1 cells triggered the recruitment of p120 RhoGTPase-activating protein/deleted in liver cancer-1 (p120RhoGAP/DLC1) to Cav1 and counteracted CagA-induced cytoskeletal rearrangements. In human GC cell lines (MKN45, N87) and mouse stomach tissue, H. pylori down-regulated endogenous expression of Cav1 independently of CagA. Mechanistically, H. pylori activated sterol-responsive element-binding protein-1 (SREBP1) to repress transcription of the human Cav1 gene from sterol-responsive elements (SREs) in the proximal Cav1 promoter. These data suggested a protective role of Cav1 against H. pylori-induced inflammation and tissue damage. We propose that H. pylori exploits down-regulation of Cav1 to subvert the host's immune response and to promote signalling of its virulence factors in host cells

Effects of a Composite Endomycorrhizal Inoculum on Olive Cuttings under the Greenhouse Conditions

This study was carried out in a nursery to evaluate the impact of mycorrhizal fungi on the cutting's root growth, and root colonization of a Moroccan olive variety ‘Picholine Marocaine' under greenhouse conditions during 2 years of cultivation. The results revealed that the inoculation with a composite inoculum of arbuscular mycorrhizal fungi (AMF) stimulated an early root formation and high development of vegetative shoots in inoculated cuttings respectively, 35 days (50 days in the control plots) and 40 days (60 days in the control plots) after their culture. The progressive establishment of mycorrhizal symbiosis in the roots of the inoculated plants showed that the root and vegetative masses were respectively 24 g and 19.5 g two years after inoculation. The average height and the leave's number of the inoculated plants relative to the control were respectively s 42/ 12 cm and 145/12. The newly formed roots were mycorrhizal and present different structures characteristic of AMF: arbuscules, vesicles, hyphae and spores, whose frequency and intensity reached 90% and 75% two years after cuttings cultivation. The arbuscular and vesicular contents and the number of spores were 67%, 96% and 212 spores/ 100 g of soil respectively. The fourteen species of mycorrhizal fungi isolated from the rhizosphere belong to 4 genera (Glomus, Acaulospora, Gigaspora, and Scutellospora) and three families (Glomaceae, Acaulosporaceae and Gigasporacea).The Glomus genus was the most dominant (65%) followed by the Gigaspora genus (22%). Glomus intraradices, Gigaspora sp.2, Glomus versiformes are the most abundant species, their frequency of occurrence are respectively 30%, 21% and 16%

Influence of storage on the volatile profile, mechanical, optical properties and antioxidant activity of strawberry spreads made with isomaltulose

[EN] This work represents the final step of a series of studies on the formulation of strawberry products with partial replacement of sucrose by healthier sugars such as fructose and isomaltulose. Previously, quality parameters of the formulated products such as colour, texture, rheology, aromatic profile and sensory evaluation were assessed. As a final step, in the present work, the volatile profile evolution of a strawberry spread-product during 90 days of storage at room temperature (20 °C), and its relation with some physicochemical properties (aw, pH, texture and colour) and antioxidant activity as well as anthocyanin content were studied. Most of the volatile compounds modified their concentration during storage; some of them totally disappeared but 13 new compounds were formed: (methyl-2-methyl butyrate, E-2-butenal, 2-butenal-2-methyl, 2-buten-1-ol, 2-penten-1-ol, 2-etil-1-hexanol, 6-methyl-5-hepten-2-one, acetic acid, propanoic-2-methyl acid, butyric acid and butyric-2-methyl acid). Storage led to a dark pink colour and an increase in consistency and adhesiveness while the antioxidant activity considerably increased (from 18±2% to 94±2% DPPH inhibition). Levels of citric acid and pectin influenced colour, texture and antioxidant activity as well as retention and formation of aromatic compounds, especially in fructose isomaltulose products. Correlations via a PLS were found between the aromatic profile of the products after storage and some of their quality parameters such as texture, colour and antioxidant content. Future research might involve correlation and identification of specific volatiles with different quality parameters.Authors would like to thank Ministry of Science and Education's General directorate of Research (AGL2008-01745/ALI) for the financial support given to this investigation.Peinado Pardo, I.; Rosa Barbosa, EM.; Heredia Gutiérrez, AB.; Escriche Roberto, MI.; Andrés Grau, AM. (2016). Influence of storage on the volatile profile, mechanical, optical properties and antioxidant activity of strawberry spreads made with isomaltulose. Food Bioscience. 14:10-20. https://doi.org/10.1016/j.fbio.2016.02.001S10201

Substantially improved pharmacokinetics of recombinant human butyrylcholinesterase by fusion to human serum albumin

<p>Abstract</p> <p>Background</p> <p>Human butyrylcholinesterase (huBChE) has been shown to be an effective antidote against multiple LD₅₀of organophosphorus compounds. A prerequisite for such use of huBChE is a prolonged circulatory half-life. This study was undertaken to produce recombinant huBChE fused to human serum albumin (hSA) and characterize the fusion protein.</p> <p>Results</p> <p>Secretion level of the fusion protein produced <it>in vitro </it>in BHK cells was ~30 mg/liter. Transgenic mice and goats generated with the fusion constructs expressed in their milk a bioactive protein at concentrations of 0.04–1.1 g/liter. BChE activity gel staining and a size exclusion chromatography (SEC)-HPLC revealed that the fusion protein consisted of predominant dimers and some monomers. The protein was confirmed to have expected molecular mass of ~150 kDa by Western blot. The purified fusion protein produced <it>in vitro </it>was injected intravenously into juvenile pigs for pharmacokinetic study. Analysis of a series of blood samples using the Ellman assay revealed a substantial enhancement of the plasma half-life of the fusion protein (~32 h) when compared with a transgenically produced huBChE preparation containing >70% tetramer (~3 h). <it>In vitro </it>nerve agent binding and inhibition experiments indicated that the fusion protein in the milk of transgenic mice had similar inhibition characteristics compared to human plasma BChE against the nerve agents tested.</p> <p>Conclusion</p> <p>Both the pharmacokinetic study and the <it>in vitro </it>nerve agent binding and inhibition assay suggested that a fusion protein retaining both properties of huBChE and hSA is produced <it>in vitro </it>and <it>in vivo</it>. The production of the fusion protein in the milk of transgenic goats provided further evidence that sufficient quantities of BChE/hSA can be produced to serve as a cost-effective and reliable source of BChE for prophylaxis and post-exposure treatment.</p

Determinants of successful clinical networks : The conceptual framework and study protocol

Background Clinical networks are increasingly being viewed as an important strategy for increasing evidence-based practice and improving models of care, but success is variable and characteristics of networks with high impact are uncertain. This study takes advantage of the variability in the functioning and outcomes of networks supported by the Australian New South Wales (NSW) Agency for Clinical Innovation's non-mandatory model of clinical networks to investigate the factors that contribute to the success of clinical networks. Methods/Design The objective of this retrospective study is to examine the association between external support, organisational and program factors, and indicators of success among 19 clinical networks over a three-year period (2006-2008). The outcomes (health impact, system impact, programs implemented, engagement, user perception, and financial leverage) and explanatory factors will be collected using a web-based survey, interviews, and record review. An independent expert panel will provide judgements about the impact or extent of each network's initiatives on health and system impacts. The ratings of the expert panel will be the outcome used in multivariable analyses. Following the rating of network success, a qualitative study will be conducted to provide a more in-depth examination of the most successful networks. Discussion This is the first study to combine quantitative and qualitative methods to examine the factors that contribute to the success of clinical networks and, more generally, is the largest study of clinical networks undertaken. The adaptation of expert panel methods to rate the impacts of networks is the methodological innovation of this study. The proposed project will identify the conditions that should be established or encouraged by agencies developing clinical networks and will be of immediate use in forming strategies and programs to maximise the effectiveness of such networks

Search for an interaction mediated by axion-like particles with ultracold neutrons at the PSI

We report on a search for a new, short-range, spin-dependent interaction using a modified version of the experimental apparatus used to measure the permanent neutron electric dipole moment at the Paul Scherrer Institute. This interaction, which could be mediated by axion-like particles, concerned the unpolarized nucleons (protons and neutrons) near the material surfaces of the apparatus and polarized ultracold neutrons stored in vacuum. The dominant systematic uncertainty resulting from magnetic-field gradients was controlled to an unprecedented level of approximately 4 pT/cm using an array of optically-pumped cesium vapor magnetometers and magnetic-field maps independently recorded using a dedicated measurement device. No signature of a theoretically predicted new interaction was found, and we set a new limit on the product of the scalar and the pseudoscalar couplings $g_sg_p\lambda^2 < 8.3 \times 10^{-28}\,\text{m}^2$ (95% C.L.) in a range of $5\,\mu\text{m} < \lambda < 25\,\text{mm}$ for the monopole-dipole interaction. This new result confirms and improves our previous limit by a factor of 2.7 and provides the current tightest limit obtained with free neutrons