Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

Variation in Number and Formation of Repeat Sequences in the rDNA ITS2 Region of Five Sibling Species in the Anopheles barbirostris Complex in Thailand

Repeat sequences of approximately 100 base pairs in length were found in the rDNA ITS2 region of Anopheles barbirostris van der Wulp (Diptera: Culicidae) species A1, A2, A3, A4, and An. campestris-like in the An. barbirostris complex. Variation in the number of repeats was observed among the five sibling species. Specifically, 10 repeats were observed in A1, eight in A2, A4, and campestris-like, and three in A3. Based on similarities in the sequences of the repeats, related repeats were classified into nine groups. Although A2, A4, and the campestris-like species had the same number of repeats, the ITS2 region of the three species contained different groups of repeats. Excluding the repeat sequences facilitated good alignment of the ITS2 region in the five sibling species. Phylogenetic analyses of the 95 isolines were compared with results obtained from mitochondrial genes (COI and COII). The results revealed marked differences among the five sibling species, particularly regarding the ITS2 region of A3, which was more distinct from the other four species than COI and COIL Repeat sequences in the ITS2 region of other Anopheles species retrieved from GenBank also were analyzed. New repeat sequences were found in An. beklemishevi Stegnii and Kabanova, An. crucians Wiedemann and An. funestus Giles, suggesting that the occurrence of repeat sequences in the ITS2 region are not rare in anopheline mosquitoes

Protocol for a randomized controlled study of Iyengar yoga for youth with irritable bowel syndrome

<p>Abstract</p> <p>Introduction</p> <p>Irritable bowel syndrome affects as many as 14% of high school-aged students. Symptoms include discomfort in the abdomen, along with diarrhea and/or constipation and other gastroenterological symptoms that can significantly impact quality of life and daily functioning. Emotional stress appears to exacerbate irritable bowel syndrome symptoms suggesting that mind-body interventions reducing arousal may prove beneficial. For many sufferers, symptoms can be traced to childhood and adolescence, making the early manifestation of irritable bowel syndrome important to understand. The current study will focus on young people aged 14-26 years with irritable bowel syndrome. The study will test the potential benefits of Iyengar yoga on clinical symptoms, psychospiritual functioning and visceral sensitivity. Yoga is thought to bring physical, psychological and spiritual benefits to practitioners and has been associated with reduced stress and pain. Through its focus on restoration and use of props, Iyengar yoga is especially designed to decrease arousal and promote psychospiritual resources in physically compromised individuals. An extensive and standardized teacher-training program support Iyengar yoga's reliability and safety. It is hypothesized that yoga will be feasible with less than 20% attrition; and the yoga group will demonstrate significantly improved outcomes compared to controls, with physiological and psychospiritual mechanisms contributing to improvements.</p> <p>Methods/Design</p> <p>Sixty irritable bowel syndrome patients aged 14-26 will be randomly assigned to a standardized 6-week twice weekly Iyengar yoga group-based program or a wait-list usual care control group. The groups will be compared on the primary clinical outcomes of irritable bowel syndrome symptoms, quality of life and global improvement at post-treatment and 2-month follow-up. Secondary outcomes will include visceral pain sensitivity assessed with a standardized laboratory task (water load task), functional disability and psychospiritual variables including catastrophizing, self-efficacy, mood, acceptance and mindfulness. Mechanisms of action involved in the proposed beneficial effects of yoga upon clinical outcomes will be explored, and include the mediating effects of visceral sensitivity, increased psychospiritual resources, regulated autonomic nervous system responses and regulated hormonal stress response assessed via salivary cortisol.</p> <p>Trial registration</p> <p>ClinicalTrials.gov <a href="http://www.clinicaltrials.gov/ct2/show/NCT01107977">NCT01107977</a>.</p

Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation