Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

Evidence for the h_b(1P) meson in the decay Upsilon(3S) --> pi0 h_b(1P)

Using a sample of 122 million Upsilon(3S) events recorded with the BaBar detector at the PEP-II asymmetric-energy e+e- collider at SLAC, we search for the $h_b(1P)$ spin-singlet partner of the P-wave chi_{bJ}(1P) states in the sequential decay Upsilon(3S) --> pi0 h_b(1P), h_b(1P) --> gamma eta_b(1S). We observe an excess of events above background in the distribution of the recoil mass against the pi0 at mass 9902 +/- 4(stat.) +/- 2(syst.) MeV/c^2. The width of the observed signal is consistent with experimental resolution, and its significance is 3.1sigma, including systematic uncertainties. We obtain the value (4.3 +/- 1.1(stat.) +/- 0.9(syst.)) x 10^{-4} for the product branching fraction BF(Upsilon(3S)-->pi0 h_b) x BF(h_b-->gamma eta_b).Comment: 8 pages, 4 postscript figures, submitted to Phys. Rev. D (Rapid Communications

The evolution and storage of primitive melts in the Eastern Volcanic Zone of Iceland: the 10 ka Grímsvötn tephra series (i.e. the Saksunarvatn ash)

Major, trace and volatile elements were measured in a suite of primitive macrocrysts and melt inclusions from the thickest layer of the 10 ka Grímsvötn tephra series (i.e. Saksunarvatn ash) at Lake Hvítárvatn in central Iceland. In the absence of primitive tholeiitic eruptions (MgO > 7 wt.%) within the Eastern Volcanic Zone (EVZ) of Iceland, these crystal and inclusion compositions provide an important insight into magmatic processes in this volcanically productive region. Matrix glass compositions show strong similarities with glass compositions from the AD 1783–84 Laki eruption, confirming the affinity of the tephra series with the Grímsvötn volcanic system. Macrocrysts can be divided into a primitive assemblage of zoned macrocryst cores (An_78–An_92, Mg#_cpx = 82–87, Fo_79.5–Fo_87) and an evolved assemblage consisting of unzoned macrocrysts and the rims of zoned macrocrysts (An_60–An_68, Mg#_cpx = 71–78, Fo_70–Fo_76). Although the evolved assemblage is close to being in equilibrium with the matrix glass, trace element disequilibrium between primitive and evolved assemblages indicates that they were derived from different distributions of mantle melt compositions. Juxtaposition of disequilibrium assemblages probably occurred during disaggregation of incompatible trace element-depleted mushes (mean La/Yb_melt = 2.1) into aphyric and incompatible trace element-enriched liquids (La/Yb_melt = 3.6) shortly before the growth of the evolved macrocryst assemblage. Post-entrapment modification of plagioclase-hosted melt inclusions has been minimal and high-Mg# inclusions record differentiation and mixing of compositionally variable mantle melts that are amongst the most primitive liquids known from the EVZ. Coupled high field strength element (HFSE) depletion and incompatible trace element enrichment in a subset of primitive plagioclase-hosted melt inclusions can be accounted for by inclusion formation following plagioclase dissolution driven by interaction with plagioclase-undersaturated melts. Thermobarometric calculations indicate that final crystal-melt equilibration within the evolved assemblage occurred at ~1140°C and 0.0–1.5 kbar. Considering the large volume of the erupted tephra and textural evidence for rapid crystallisation of the evolved assemblage, 0.0–1.5 kbar is considered unlikely to represent a pressure of long-term magma accumulation and storage. Multiple thermometers indicate that the primitive assemblage crystallised at high temperatures of 1240–1300°C. Different barometers, however, return markedly different crystallisation depth estimates. Raw clinopyroxene-melt pressures of 5.5–7.5 kbar conflict with apparent melt inclusion entrapment pressures of 1.4 kbar. After applying a correction derived from published experimental data, clinopyroxene-melt equilibria return mid-crustal pressures of 4±1.5 kbar, which are consistent with pressures estimated from the major element content of primitive melt inclusions. Long-term storage of primitive magmas in the mid-crust implies that low CO_2 concentrations measured in primitive plagioclase-hosted inclusions (262–800 ppm) result from post-entrapment CO_2 loss during transport through the shallow crust. In order to reconstruct basaltic plumbing system geometries from petrological data with greater confidence, mineral-melt equilibrium models require refinement at pressures of magma storage in Iceland. Further basalt phase equilibria experiments are thus needed within the crucial 1–7 kbar range.D.A.N. was supported by a Natural Environment Research Council studentship (NE/1528277/1) at the start of this project. SIMS analyses were supported by Natural Environment Research Council Ion Microprobe Facility award (IMF508/1013).This is the final version of the article. It first appeared from Springer via http://dx.doi.org/10.1007/s00410-015-1170-

Neutrinos

229 pages229 pages229 pagesThe Proceedings of the 2011 workshop on Fundamental Physics at the Intensity Frontier. Science opportunities at the intensity frontier are identified and described in the areas of heavy quarks, charged leptons, neutrinos, proton decay, new light weakly-coupled particles, and nucleons, nuclei, and atoms

The diagnostic value of endoscopy and Helicobacter pylori tests for peptic ulcer patients in late post-treatment setting

BACKGROUND: Guidelines for management of peptic ulcer patients after the treatment are largely directed to detection of H. pylori infection using only non-invasive tests. We compared the diagnostic value of non-invasive and endoscopy based H. pylori tests in a late post-treatment setting. METHODS: Altogether 34 patients with dyspeptic complaints were referred for gastroscopy 5 years after the treatment of peptic ulcer using a one-week triple therapy scheme. The endoscopic and histologic findings were evaluated according to the Sydney classification. Bacteriological, PCR and cytological investigations and (13)C-UBT tests were performed. RESULTS: Seventeen patients were defined H. pylori positive by (13)C-UBT test, PCR and histological examination. On endoscopy, peptic ulcer persisted in 4 H. pylori positive cases. Among the 6 cases with erosions of the gastric mucosa, only two patients were H. pylori positive. Mucosal atrophy and intestinal metaplasia were revealed both in the H. pylori positive and H. pylori negative cases. Bacteriological examination revealed three clarithromycin resistant H. pylori strains. Cytology failed to prove validity for diagnosing H. pylori in a post-treatment setting. CONCLUSIONS: In a late post-treatment setting, patients with dyspepsia should not be monitored only by non-invasive investigation methods; it is also justified to use the classical histological evaluation of H. pylori colonisation, PCR and bacteriology as they have shown good concordance with (13)C-UBT. Moreover, endoscopy and histological investigation of a gastric biopsy have proved to be the methods with an additional diagnostic value, providing the physician with information about inflammatory, atrophic and metaplastic lesions of the stomach in dyspeptic H. pylori positive and negative patients. Bacteriological methods are suggested for detecting the putative antimicrobial resistance of H. pylori, aimed at successful eradication of infection in persistent peptic ulcer cases

Review of epidemiologic data on the debate over smokeless tobacco's role in harm reduction

Some tobacco researchers have argued that the European Union should remove its ban on a form of low-nitrosamine smokeless tobacco referred to as Swedish 'snus'. This argument has developed in to an international debate over the use of smokeless tobacco as a measure of harm reduction for smokers. Leading authorities in the USA have firmly stated that there is no safe tobacco - a message which does not allow for any discussion of comparative tobacco risks. This commentary is intended to review the origin of the controversy over Swedish 'snus', to examine briefly the meta-analysis on cancer risks by Peter Lee and Jan Hamling (published in July in BMC Medicine) and to discuss the anticipated direction of the debate on tobacco-harm reduction in the USA. We anticipate that much of the debate will shift from the discussion of epidemiologic data to the discussion of the marketing, health communication and economics of smokeless tobacco. While the Food and Drug Administration's newly approved authority over tobacco will undoubtedly affect the smokeless products, it may not be the sole determinant of harm reduction's fate in the USA

Additional Serine/Threonine Phosphorylation Reduces Binding Affinity but Preserves Interface Topography of Substrate Proteins to the c-Cbl TKB Domain

The E3-ubiquitin ligase, c-Cbl, is a multi-functional scaffolding protein that plays a pivotal role in controlling cell phenotype. As part of the ubiquitination and downregulation process, c-Cbl recognizes targets, such as tyrosine kinases and the Sprouty proteins, by binding to a conserved (NX/R)pY(S/T)XXP motif via its uniquely embedded SH2 domain (TKB domain). We previously outlined the mode of binding between the TKB domain and various substrate peptide motifs, including epidermal growth factor receptor (EGFR) and Sprouty2 (Spry2), and demonstrated that an intrapetidyl hydrogen bond forms between the (pY-1) arginine or (pY-2) asparagine and the phosphorylated tyrosine, which is crucial for binding. Recent reports demonstrated that, under certain types of stimulation, the serine/threonine residues at the pY+1 and/or pY+2 positions within this recognition motif of EGFR and Sprouty2 may be endogenously phosphorylated. Using structural and binding studies, we sought to determine whether this additional phosphorylation could affect the binding of the TKB domain to these peptides and consequently, whether the type of stimulation can dictate the degree to which substrates bind to c-Cbl. Here, we show that additional phosphorylation significantly reduces the binding affinity between the TKB domain and its target proteins, EGFR and Sprouty2, as compared to peptides bearing a single tyrosine phosphorylation. The crystal structure indicates that this is accomplished with minimal changes to the essential intrapeptidyl bond and that the reduced strength of the interaction is due to the charge repulsion between c-Cbl and the additional phosphate group. This obvious reduction in binding affinity, however, indicates that Cbl's interactions with its TKB-centered binding partners may be more favorable in the absence of Ser/Thr phosphorylation, which is stimulation and context specific in vivo. These results demonstrate the importance of understanding the environment in which certain residues are phosphorylated, and the necessity of including this in structural investigations

Host-derived RANKL is responsible for osteolysis in a C4-2 human prostate cancer xenograft model of experimental bone metastases

<p>Abstract</p> <p>Background</p> <p>C4-2 prostate cancer (CaP) cells grown in mouse tibiae cause a mixed osteoblastic/osteolytic response with increases in osteoclast numbers and bone resorption. Administration of osteoprotegerin (OPG) blocks these increases, indicating the critical role of RANKL in osteolysis in this model. The objective of our study was to investigate whether RANKL expressed by tumor cells (human origin) directly stimulates osteolysis associated with the growth of these cells in bone or whether the increased osteolysis is caused by RANKL expressed by the host environment cells (murine origin). The relative contribution of tumor-<it>vs. </it>host-derived RANKL has been difficult to establish, even with human xenografts, because murine and human RANKL are both capable of stimulating osteolysis in mice, and the RANKL inhibitors used to date (OPG and RANK-Fc) inhibit human and murine RANKL.</p> <p>Methods</p> <p>To address this question we used a neutralizing, antibody (huRANKL MAb), which specifically neutralizes the biological activities of human RANKL and thereby the contribution of C4-2 derived RANKL in this tibial injection model of experimental bone metastases.</p> <p>Results</p> <p>Administration of huRANKL MAb did not inhibit the osteolytic response of the bone to these cells, or affect the establishment and growth of the C4-2 tumors in this environment.</p> <p>Conclusion</p> <p>In conclusion, our results suggest that in this model, murine RANKL and not the tumor-derived human RANKL is the mediator of the osteolytic reaction associated with C4-2 growth in bone. We hypothesize that C4-2 cells express other factor/s inducing host production of RANKL, thereby driving tumor-associated osteolysis.</p

Effect of a vitamin/mineral supplement on children and adults with autism

<p>Abstract</p> <p>Background</p> <p>Vitamin/mineral supplements are among the most commonly used treatments for autism, but the research on their use for treating autism has been limited.</p> <p>Method</p> <p>This study is a randomized, double-blind, placebo-controlled three month vitamin/mineral treatment study. The study involved 141 children and adults with autism, and pre and post symptoms of autism were assessed. None of the participants had taken a vitamin/mineral supplement in the two months prior to the start of the study. For a subset of the participants (53 children ages 5-16) pre and post measurements of nutritional and metabolic status were also conducted.</p> <p>Results</p> <p>The vitamin/mineral supplement was generally well-tolerated, and individually titrated to optimum benefit. Levels of many vitamins, minerals, and biomarkers improved/increased showing good compliance and absorption. Statistically significant improvements in metabolic status were many including: total sulfate (+17%, p = 0.001), S-adenosylmethionine (SAM; +6%, p = 0.003), reduced glutathione (+17%, p = 0.0008), ratio of oxidized glutathione to reduced glutathione (GSSG:GSH; -27%, p = 0.002), nitrotyrosine (-29%, p = 0.004), ATP (+25%, p = 0.000001), NADH (+28%, p = 0.0002), and NADPH (+30%, p = 0.001). Most of these metabolic biomarkers improved to normal or near-normal levels.</p> <p>The supplement group had significantly greater improvements than the placebo group on the Parental Global Impressions-Revised (PGI-R, Average Change, p = 0.008), and on the subscores for Hyperactivity (p = 0.003), Tantrumming (p = 0.009), Overall (p = 0.02), and Receptive Language (p = 0.03). For the other three assessment tools the difference between treatment group and placebo group was not statistically significant.</p> <p>Regression analysis revealed that the degree of improvement on the Average Change of the PGI-R was strongly associated with several biomarkers (adj. R²= 0.61, p < 0.0005) with the initial levels of biotin and vitamin K being the most significant (p < 0.05); both biotin and vitamin K are made by beneficial intestinal flora.</p> <p>Conclusions</p> <p>Oral vitamin/mineral supplementation is beneficial in improving the nutritional and metabolic status of children with autism, including improvements in methylation, glutathione, oxidative stress, sulfation, ATP, NADH, and NADPH. The supplement group had significantly greater improvements than did the placebo group on the PGI-R Average Change. This suggests that a vitamin/mineral supplement is a reasonable adjunct therapy to consider for most children and adults with autism.</p> <p>Trial Registration</p> <p><b>Clinical Trial Registration Number: </b><a href="http://www.clinicaltrials.gov/ct2/show/NCT01225198">NCT01225198</a></p

In Patients with Established RA, Positive Effects of a Randomised Three Month WBV Therapy Intervention on Functional Ability, Bone Mineral Density and Fatigue Are Sustained for up to Six Months

Functional ability is often impaired for people with rheumatoid arthritis (RA), rendering these patients highly sedentary. Additionally, patients with RA often take medication known to negatively affect bone mass. Thus improving functional ability and bone health in this group of patients is important. The aim of this study was to investigate the effects of whole body vibration (WBV) therapy in patients with stable, established RA. Thirty one females with RA were randomly assigned to a control group (CON, n = 15) who continued with their normal activities or a WBV group (n = 16) who underwent a three month WBV therapy intervention, consisting of 15 minutes of intermittent vibration, performed twice per week. Patients were assessed at baseline, three months, and three months post intervention for functional ability using the modified Health Assessment Questionnaire; for RA disease activity using the Clinical Disease Activity Index, for quality of life using self-report fatigue and pain scores; for physical activity profiles using accelerometry, and for BMD and body composition using DXA. Patients in both groups were matched for all variables at baseline. After the intervention period, functional ability was significantly improved in the WBV group (1.22(0.19) to 0.92(0.19), p = 0.02). Hip BMD was significantly reduced in the CON group (0.97(0.05) to 0.84(0.05) g.cm-2, p = 0.01), while no decreases were seen in the WBV group (1.01(0.05) to 0.94(0.05) g.cm-2, p = 0.50). Despite no change in RA disease activity in either group at either follow up, fatigue levels were improved in the WBV group (4.4(0.63) to 1.1(0.65), yet remained unchanged in the CON group at both follow ups (p = 0.01). Ten minute bouts of light to moderate physical activity were significantly reduced in the CON group after the intervention (2.8(0.61) to 1.8(0.64) bouts per day, p = 0.01), and were preserved in the WBV group (3.1(0.59) to 3.0(0.61) bouts per day, p = 0.70). Intermittent WBV shows promise for sustained improvements in functional ability, for attenuating loss of bone mass at the hip, as well as for decreasing fatigue in patients with established RA. TRIAL REGISTRATION: Pan African Clinical Trials Registry PACTR201405000823418