Branching Time Active Inference: empirical study and complexity class analysis

Active inference is a state-of-the-art framework for modelling the brain that explains a wide range of mechanisms such as habit formation, dopaminergic discharge and curiosity. However, recent implementations suffer from an exponential complexity class when computing the prior over all the possible policies up to the time horizon. Fountas et al (2020) used Monte Carlo tree search to address this problem, leading to very good results in two different tasks. Additionally, Champion et al (2021a) proposed a tree search approach based on (temporal) structure learning. This was enabled by the development of a variational message passing approach to active inference, which enables compositional construction of Bayesian networks for active inference. However, this message passing tree search approach, which we call branching-time active inference (BTAI), has never been tested empirically. In this paper, we present an experimental study of BTAI in the context of a maze solving agent. In this context, we show that both improved prior preferences and deeper search help mitigate the vulnerability to local minima. Then, we compare BTAI to standard active inference (AcI) on a graph navigation task. We show that for small graphs, both BTAI and AcI successfully solve the task. For larger graphs, AcI exhibits an exponential (space) complexity class, making the approach intractable. However, BTAI explores the space of policies more efficiently, successfully scaling to larger graphs. Then, BTAI was compared to the POMCP algorithm on the frozen lake environment. The experiments suggest that BTAI and the POMCP algorithm accumulate a similar amount of reward. Also, we describe when BTAI receives more rewards than the POMCP agent, and when the opposite is true. Finally, we compared BTAI to the approach of Fountas et al (2020) on the dSprites dataset, and we discussed the pros and cons of each approach.Comment: 39 pages, 11 figures, accepted for publication in Neural Network

Branching Time Active Inference: empirical study and complexity class analysis

Active inference is a state-of-the-art framework for modelling the brain that explains a wide range of mechanisms such as habit formation, dopaminergic discharge and curiosity. However, recent implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to very good results in two different tasks. Additionally, Champion et al. (2021a) proposed a tree search approach based on (temporal) structure learning. This was enabled by the development of a variational message passing approach to active inference (Champion, Bowman, Grześ, 2021), which enables compositional construction of Bayesian networks for active inference. However, this message passing tree search approach, which we call branching-time active inference (BTAI), has never been tested empirically. In this paper, we present an experimental study of the approach (Champion, Grześ, Bowman, 2021) in the context of a maze solving agent. In this context, we show that both improved prior preferences and deeper search help mitigate the vulnerability to local minima. Then, we compare BTAI to standard active inference (AcI) on a graph navigation task. We show that for small graphs, both BTAI and AcI successfully solve the task. For larger graphs, AcI exhibits an exponential (space) complexity class, making the approach intractable. However, BTAI explores the space of policies more efficiently, successfully scaling to larger graphs. Then, BTAI was compared to the POMCP algorithm (Silver and Veness, 2010) on the frozen lake environment. The experiments suggest that BTAI and the POMCP algorithm accumulate a similar amount of reward. Also, we describe when BTAI receives more rewards than the POMCP agent, and when the opposite is true. Finally, we compared BTAI to the approach of Fountas et al. (2020) on the dSprites dataset, and we discussed the pros and cons of each approach

Realising Active Inference in Variational Message Passing: the Outcome-blind Certainty Seeker

Active inference is a state-of-the-art framework in neuroscience that offers a unified theoryof brain function. It is also proposed as a framework for planning in AI. Unfortunately, thecomplex mathematics required to create new models — can impede application of activeinference in neuroscience and AI research. This paper addresses this problem by providinga complete mathematical treatment of the active inference framework — in discrete timeand state spaces — and the derivation of the update equations for any new model. Weleverage the theoretical connection between active inference and variational message passingas describe by John Winn and Christopher M. Bishop in 2005. Since, variational messagepassing is a well-defined methodology for deriving Bayesian belief update equations, thispaper opens the door to advanced generative models for active inference. We show thatusing a fully factorized variational distribution simplifies the expected free energy — that furnishes priors over policies — so that agents seek unambiguous states. Finally, we considerfuture extensions that support deep tree searches for sequential policy optimisation — basedupon structure learning and belief propagation

Branching Time Active Inference with Bayesian Filtering

Branching Time Active Inference (Champion et al., 2022b,a) is a framework proposing to look at planning as a form of Bayesian model expansion. Its root can be found in Active Inference (Friston et al., 2016; Da Costa et al., 2020; Champion et al., 2021), a neuroscientific framework widely used for brain modelling, as well as in Monte Carlo Tree Search (Browne et al., 2012), a method broadly applied in the Reinforcement Learning literature. Up to now, the inference of the latent variables was carried out by taking advantage of the flexibility offered by Variational Message Passing (Winn and Bishop, 2005), an iterative process that can be understood as sending messages along the edges of a factor graph (Forney, 2001). In this paper, we harness the efficiency of an alternative method for inference called Bayesian Filtering (Fox et al., 2003), which does not require the iteration of the update equations until convergence of the Variational Free Energy. Instead, this scheme alternates between two phases: integration of evidence and prediction of future states. Both of those phases can be performed efficiently and this provides a forty times speed up over the state-of-the-art

Branching Time Active Inference: The theory and its generality

Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies

Characterization of water and wildlife strains as a subgroup of Campylobacter jejuni using DNA microarrays.

Campylobacter jejuni is the leading cause of human bacterial gastroenteritis worldwide, but source attribution of the organism is difficult. Previously, DNA microarrays were used to investigate isolate source, which suggested a non-livestock source of infection. In this study we analysed the genome content of 162 clinical, livestock and water and wildlife (WW) associated isolates combined with the previous study. Isolates were grouped by genotypes into nine clusters (C1 to C9). Multilocus sequence typing (MLST) data demonstrated that livestock associated clonal complexes dominated clusters C1-C6. The majority of WW isolates were present in the C9 cluster. Analysis of previously reported genomic variable regions demonstrated that these regions were linked to specific clusters. Two novel variable regions were identified. A six gene multiplex PCR (mPCR) assay, designed to effectively differentiated strains into clusters, was validated with 30 isolates. A further five WW isolates were tested by mPCR and were assigned to the C7-C9 group of clusters. The predictive mPCR test could be used to indicate if a clinical case has come from domesticated or WW sources. Our findings provide further evidence that WW C. jejuni subtypes show niche adaptation and may be important in causing human infection

An Online Dynamic Amplitude-Correcting Gradient Estimation Technique to Align X-ray Focusing Optics

High-brightness X-ray pulses, as generated at synchrotrons and X-ray free electron lasers (XFEL), are used in a variety of scientific experiments. Many experimental testbeds require optical equipment, e.g Compound Refractive Lenses (CRLs), to be precisely aligned and focused. The lateral alignment of CRLs to a beamline requires precise positioning along four axes: two translational, and the two rotational. At a synchrotron, alignment is often accomplished manually. However, XFEL beamlines present a beam brightness that fluctuates in time, making manual alignment a time-consuming endeavor. Automation using classic stochastic methods often fail, given the errant gradient estimates. We present an online correction based on the combination of a generalized finite difference stencil and a time-dependent sampling pattern. Error expectation is analyzed, and efficacy is demonstrated. We provide a proof of concept by laterally aligning optics on a simulated XFEL beamline

Impact of Trauma System Structure on Injury Outcomes : A Systematic Review and Meta-Analysis

The effectiveness of trauma systems in decreasing injury mortality and morbidity has been well demonstrated. However, little is known about which components contribute to their effectiveness. We aimed to systematically review the evidence of the impact of trauma system components on clinically important injury outcomes. We searched MEDLINE, EMBASE, Cochrane CENTRAL, and BIOSIS/Web of Knowledge, gray literature and trauma association Web sites to identify studies evaluating the association between at least one trauma system component and injury outcome. We calculated pooled effect estimates using inverse-variance random-effects models. We evaluated quality of evidence using GRADE criteria. We screened 15,974 records, retaining 41 studies for qualitative synthesis and 19 for meta-analysis. Two recommended trauma system components were associated with reduced odds of mortality: inclusive design (odds ratio [OR] = 0.72 [0.65-0.80]) and helicopter transport (OR = 0.70 [0.55-0.88]). Pre-Hospital Advanced Trauma Life Support was associated with a significant reduction in hospital days (mean difference [MD] = 5.7 [4.4-7.0]) but a nonsignificant reduction in mortality (OR = 0.78 [0.44-1.39]). Population density of surgeons was associated with a nonsignificant decrease in mortality (MD = 0.58 [-0.22 to 1.39]). Trauma system maturity was associated with a significant reduction in mortality (OR = 0.76 [0.68-0.85]). Quality of evidence was low or very low for mortality and healthcare utilization. This review offers low-quality evidence for the effectiveness of an inclusive design and trauma system maturity and very-low-quality evidence for helicopter transport in reducing injury mortality. Further research should evaluate other recommended components of trauma systems and non-fatal outcomes and explore the impact of system component interactions.Peer reviewe

Impact of Trauma System Structure on Injury Outcomes : A Systematic Review and Meta-Analysis

The effectiveness of trauma systems in decreasing injury mortality and morbidity has been well demonstrated. However, little is known about which components contribute to their effectiveness. We aimed to systematically review the evidence of the impact of trauma system components on clinically important injury outcomes. We searched MEDLINE, EMBASE, Cochrane CENTRAL, and BIOSIS/Web of Knowledge, gray literature and trauma association Web sites to identify studies evaluating the association between at least one trauma system component and injury outcome. We calculated pooled effect estimates using inverse-variance random-effects models. We evaluated quality of evidence using GRADE criteria. We screened 15,974 records, retaining 41 studies for qualitative synthesis and 19 for meta-analysis. Two recommended trauma system components were associated with reduced odds of mortality: inclusive design (odds ratio [OR] = 0.72 [0.65-0.80]) and helicopter transport (OR = 0.70 [0.55-0.88]). Pre-Hospital Advanced Trauma Life Support was associated with a significant reduction in hospital days (mean difference [MD] = 5.7 [4.4-7.0]) but a nonsignificant reduction in mortality (OR = 0.78 [0.44-1.39]). Population density of surgeons was associated with a nonsignificant decrease in mortality (MD = 0.58 [-0.22 to 1.39]). Trauma system maturity was associated with a significant reduction in mortality (OR = 0.76 [0.68-0.85]). Quality of evidence was low or very low for mortality and healthcare utilization. This review offers low-quality evidence for the effectiveness of an inclusive design and trauma system maturity and very-low-quality evidence for helicopter transport in reducing injury mortality. Further research should evaluate other recommended components of trauma systems and non-fatal outcomes and explore the impact of system component interactions.Peer reviewe

The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: an evidence synthesis.

BACKGROUND: Child maltreatment is a substantial social problem that affects large numbers of children and young people in the UK, resulting in a range of significant short- and long-term psychosocial problems. OBJECTIVES: To synthesise evidence of the effectiveness, cost-effectiveness and acceptability of interventions addressing the adverse consequences of child maltreatment. STUDY DESIGN: For effectiveness, we included any controlled study. Other study designs were considered for economic decision modelling. For acceptability, we included any study that asked participants for their views. PARTICIPANTS: Children and young people up to 24 years 11 months, who had experienced maltreatment before the age of 17 years 11 months. INTERVENTIONS: Any psychosocial intervention provided in any setting aiming to address the consequences of maltreatment. MAIN OUTCOME MEASURES: Psychological distress [particularly post-traumatic stress disorder (PTSD), depression and anxiety, and self-harm], behaviour, social functioning, quality of life and acceptability. METHODS: Young Persons and Professional Advisory Groups guided the project, which was conducted in accordance with Cochrane Collaboration and NHS Centre for Reviews and Dissemination guidance. Departures from the published protocol were recorded and explained. Meta-analyses and cost-effectiveness analyses of available data were undertaken where possible. RESULTS: We identified 198 effectiveness studies (including 62 randomised trials); six economic evaluations (five using trial data and one decision-analytic model); and 73 studies investigating treatment acceptability. Pooled data on cognitive-behavioural therapy (CBT) for sexual abuse suggested post-treatment reductions in PTSD [standardised mean difference (SMD) -0.44 (95% CI -4.43 to -1.53)], depression [mean difference -2.83 (95% CI -4.53 to -1.13)] and anxiety [SMD -0.23 (95% CI -0.03 to -0.42)]. No differences were observed for post-treatment sexualised behaviour, externalising behaviour, behaviour management skills of parents, or parental support to the child. Findings from attachment-focused interventions suggested improvements in secure attachment [odds ratio 0.14 (95% CI 0.03 to 0.70)] and reductions in disorganised behaviour [SMD 0.23 (95% CI 0.13 to 0.42)], but no differences in avoidant attachment or externalising behaviour. Few studies addressed the role of caregivers, or the impact of the therapist-child relationship. Economic evaluations suffered methodological limitations and provided conflicting results. As a result, decision-analytic modelling was not possible, but cost-effectiveness analysis using effectiveness data from meta-analyses was undertaken for the most promising intervention: CBT for sexual abuse. Analyses of the cost-effectiveness of CBT were limited by the lack of cost data beyond the cost of CBT itself. CONCLUSIONS: It is not possible to draw firm conclusions about which interventions are effective for children with different maltreatment profiles, which are of no benefit or are harmful, and which factors encourage people to seek therapy, accept the offer of therapy and actively engage with therapy. Little is known about the cost-effectiveness of alternative interventions. LIMITATIONS: Studies were largely conducted outside the UK. The heterogeneity of outcomes and measures seriously impacted on the ability to conduct meta-analyses. FUTURE WORK: Studies are needed that assess the effectiveness of interventions within a UK context, which address the wider effects of maltreatment, as well as specific clinical outcomes. STUDY REGISTRATION: This study is registered as PROSPERO CRD42013003889. FUNDING: The National Institute for Health Research Health Technology Assessment programme