Active Inference: Demystified and Compared

Active inference is a first principle account of how autonomous agents operate in dynamic, nonstationary environments. This problem is also considered in reinforcement learning, but limited work exists on comparing the two approaches on the same discrete-state environments. In this letter, we provide (1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in reinforcement learning, and (2) an explicit discrete-state comparison between active inference and reinforcement learning on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of reinforcement learning. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration-and account for uncertainty about their environment-in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in reinforcement learning is removed in active inference, where reward can simply be treated as another observation we have a preference over; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based reinforcement learning agents and by placing zero prior preferences over rewards and learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings (e.g., robotic arm movement, Atari games) if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation and demonstrate these behaviors in a OpenAI gym environment, alongside reinforcement learning agents

Constraints for the nuclear parton distributions from Z and W production at the LHC

The LHC is foreseen to finally bring also the nuclear collisions to the TeV scale thereby providing new possibilities for physics studies, in particular related to the electro-weak sector of the Standard Model. We study here the Z and W production in proton-lead and lead-lead collisions at the LHC, concentrating on the prospects of testing the factorization and constraining the nuclear modifications of the parton distribution functions (PDFs). Especially, we find that the rapidity asymmetries in proton-nucleus collisions, arising from the differences in the PDFs between the colliding objects, provide a decisive advantage in comparison to the rapidity-symmetric nucleus-nucleus case. We comment on how such studies will help to improve our knowledge of the nuclear PDFs.Comment: The version accepted for publication in JHEP. New figures has been added, and we also discuss the single charged lepton productio

Obesity prevention in early life: An opportunity to better support the role of Maternal and Child Health Nurses in Australia

© 2015 Laws et al.; licensee BioMed Central. Background: Because parents with young children access primary health care services frequently, a key opportunity arises for Maternal and Child Health (MCH) nurses to actively work with families to support healthy infant feeding practices and lifestyle behaviours. However, little is known regarding the extent to which MCH nurses promote obesity prevention practices and how such practices could be better supported. Methods: This mixed methods study involved a survey of 56 MCH nurses (response rate 84.8%), 16 of whom participated in semi-structured qualitative interviews. Both components aimed to examine the extent to which nurses addressed healthy infant feeding practices, healthy eating, active play and limiting sedentary behavior during routine consultations with young children 0-5 years. Key factors influencing such practices and how they could be best supported were also investigated. All data were collected from September to December 2013. Survey data were analysed descriptively and triangulated with qualitative interview findings, the analysis of which was guided by grounded theory principles. Results: Although nurses reported measuring height/length and weight in most consultations, almost one quarter (22.2 %) reported never/rarely using growth charts to identify infants or children at risk of overweight or obesity. This reflected a reluctance to raise the issue of weight with parents and a lack of confidence in how to address it. The majority of nurses reported providing advice on aspects of infant feeding relevant to obesity prevention at most consultations, with around a third (37 %) routinely provided advice on formula preparation. Less than half of nurses routinely promoted active play and only 30 % discussed limiting sedentary behaviour such as TV viewing. Concerns about parental receptiveness and maintaining rapport were key barriers to more effective implementation. Conclusion: While MCH nurses are well placed to address obesity prevention in early life, there is currently a missed public health opportunity. Improving nurse skills in behaviour change counseling will be key to increasing their confidence in raising sensitive lifestyle issues with parents to better integrate obesity prevention practices into normal MCH service delivery

The impact of interventions to prevent obesity or improve obesity related behaviours in children (0-5 years) from socioeconomically disadvantaged and/or indigenous families: A systematic review

© 2014 Laws et al.; licensee BioMed Central Ltd. Background: Children from disadvantaged families including those from low socioeconomic backgrounds and Indigenous families have higher rates of obesity, making early intervention a priority. The aim of this study was to systematically review the literature to examine the effectiveness of interventions to prevent obesity or improve obesity related behaviours in children 0-5 years from socioeconomically disadvantaged or Indigenous families. Methods. Searches of major electronic databases identified articles published from 1993-2013 targeting feeding practices, anthropometric, diet, activity or sedentary behaviour outcomes. This was supplemented with snowballing from existing reviews and primary studies. Data extraction was undertaken by one author and cross checked by another. Quality assessments included both internal and external validity. Results: Thirty-two studies were identified, with only two (both low quality) in Indigenous groups. Fourteen studies had a primary aim to prevent obesity. Mean differences between intervention and control groups ranged from -0.29 kg/m2to -0.54 kg/m2for body mass index (BMI) and -2.9 to -25.6% for the prevalence of overweight/obesity. Interventions initiated in infancy (under two years) had a positive impact on obesity related behaviours (e.g. diet quality) but few measured the longer-term impact on healthy weight gain. Findings amongst pre-schoolers (3-5 years) were mixed, with the more successful interventions requiring high levels of parental engagement, use of behaviour change techniques, a focus on skill building and links to community resources. Less than 10% of studies were high quality. Future studies should focus on improving study quality, including follow-up of longer-term anthropometric outcomes, assessments of cost effectiveness, acceptability in target populations and potential for implementation in routine service delivery. Conclusion: There is an urgent need for further research on effective obesity prevention interventions for Indigenous children. The findings from the growing body of intervention research focusing on obesity prevention amongst young children from socioeconomically disadvantaged families suggest intervention effects are modest but promising. Further high quality studies with longer term follow up are required. Trial registration. PROSPERO Registration no: CRD42013006536

Changes in use of herbs and dietary supplements (HDS) among clinicians enrolled in an online curriculum

<p>Abstract</p> <p>Background</p> <p>Little is known about clinicians' use of herbs and dietary supplements (HDS), how their personal HDS use changes with time and training, and how changes in their personal use affect their confidence or communication with patients about HDS.</p> <p>Methods</p> <p>We conducted a prospective cohort study of clinicians before and after an on-line curriculum about HDS in winter-spring, 2005.</p> <p>Results</p> <p>Of the 569 clinicians who completed surveys both at baseline and after the course, 25% were male and the average age was 42 years old; 88% used HDS before and after the course. The average number of supplements used fell slightly from 6.2 at baseline to 5.8 after the course (P < 0.01). The most commonly used supplements at baseline were: multivitamins (65%), calcium (42%), B vitamins (34%), vitamin C (34%), green tea (27%), fish oil (27%) and vitamin E (25%). Use of fish oil increased to 30% after the course (P = 0.01). Use of supplements traditionally used to treat colds decreased: vitamin C (34% to 27%), zinc (13% to 10%), and echinacea (7% to 5%, P < 0.05 for all three). Changes in personal HDS use were not associated with significant changes in confidence or communication with patients.</p> <p>Conclusion</p> <p>Many clinicians use HDS personally; use changes seasonally and to a small extent with professional education. Professional use of HDS is dynamic and seasonal. Additional research is needed to understand the impact of personal use on professional attitudes and behavior in populations with lower baseline uses of HDS.</p

Gas valves, forests and global change: a commentary on Jarvis (1976) 'The interpretation of the variations in leaf water potential and stomatal conductance found in canopies in the field'.

Microscopic turgor-operated gas valves on leaf surfaces-stomata-facilitate gas exchange between the plant and the atmosphere, and respond to multiple environmental and endogenous cues. Collectively, stomatal activities affect everything from the productivity of forests, grasslands and crops to biophysical feedbacks between land surface vegetation and climate. In 1976, plant physiologist Paul Jarvis reported an empirical model describing stomatal responses to key environmental and plant conditions that predicted the flux of water vapour from leaves into the surrounding atmosphere. Subsequent theoretical advances, building on this earlier approach, established the current paradigm for capturing the physiological behaviour of stomata that became incorporated into sophisticated models of land carbon cycling. However, these models struggle to accurately predict observed trends in the physiological responses of Northern Hemisphere forests to recent atmospheric CO2 increases, highlighting the need for improved representation of the role of stomata in regulating forest-climate interactions. Bridging this gap between observations and theory as atmospheric CO2 rises and climate change accelerates creates challenging opportunities for the next generation of physiologists to advance planetary ecology and climate science. This commentary was written to celebrate the 350th anniversary of the journal Philosophical Transactions of the Royal Society

Epidemics on contact networks: a general stochastic approach

Dynamics on networks is considered from the perspective of Markov stochastic processes. We partially describe the state of the system through network motifs and infer any missing data using the available information. This versatile approach is especially well adapted for modelling spreading processes and/or population dynamics. In particular, the generality of our systematic framework and the fact that its assumptions are explicitly stated suggests that it could be used as a common ground for comparing existing epidemics models too complex for direct comparison, such as agent-based computer simulations. We provide many examples for the special cases of susceptible-infectious-susceptible (SIS) and susceptible-infectious-removed (SIR) dynamics (e.g., epidemics propagation) and we observe multiple situations where accurate results may be obtained at low computational cost. Our perspective reveals a subtle balance between the complex requirements of a realistic model and its basic assumptions.Comment: Main document: 16 pages, 7 figures. Electronic Supplementary Material (included): 6 pages, 1 tabl

Serotonin tranporter methylation and response to cognitive behaviour therapy in children with anxiety disorders

Anxiety disorders that are the most commonly occurring psychiatric disorders in childhood, are associated with a range of social and educational impairments and often continue into adulthood. Cognitive behaviour therapy (CBT) is an effective treatment option for the majority of cases, although up to 35-45% of children do not achieve remission. Recent research suggests that some genetic variants may be associated with a more beneficial response to psychological therapy. Epigenetic mechanisms such as DNA methylation work at the interface between genetic and environmental influences. Furthermore, epigenetic alterations at the serotonin transporter (SERT) promoter region have been associated with environmental influences such as stressful life experiences. In this study, we measured DNA methylation upstream of SERT in 116 children with an anxiety disorder, before and after receiving CBT. Change during treatment in percentage DNA methylation was significantly different in treatment responders vs nonresponders. This effect was driven by one CpG site in particular, at which responders increased in methylation, whereas nonresponders showed a decrease in DNA methylation. This is the first study to demonstrate differences in SERT methylation change in association with response to a purely psychological therapy. These findings confirm that biological changes occur alongside changes in symptomatology following a psychological therapy such as CBT

Association between fast food purchasing and the local food environment

Objective: In this study, an instrument was created to measure the healthy and unhealthy characteristics of food environments and investigate associations between the whole of the food environment and fast food consumption.Design and subjects: In consultation with other academic researchers in this field, food stores were categorised to either healthy or unhealthy and weighted (between +10 and &minus;10) by their likely contribution to healthy/unhealthy eating practices. A healthy and unhealthy food environment score (FES) was created using these weightings. Using a cross-sectional study design, multilevel multinomial regression was used to estimate the effects of the whole food environment on the fast food purchasing habits of 2547 individuals.Results: Respondents in areas with the highest tertile of the healthy FES had a lower likelihood of purchasing fast food both infrequently and frequently compared with respondents who never purchased, however only infrequent purchasing remained significant when simultaneously modelled with the unhealthy FES (odds ratio (OR) 0.52; 95% confidence interval (CI) 0.32&ndash;0.83). Although a lower likelihood of frequent fast food purchasing was also associated with living in the highest tertile of the unhealthy FES, no association remained once the healthy FES was included in the models. In our binary models, respondents living in areas with a higher unhealthy FES than healthy FES were more likely to purchase fast food infrequently (OR 1.35; 95% CI 1.00&ndash;1.82) however no association was found for frequent purchasing.Conclusion: Our study provides some evidence to suggest that healthier food environments may discourage fast food purchasing.<br /