Active Inference: Demystified and Compared

Active inference is a first principle account of how autonomous agents operate in dynamic, nonstationary environments. This problem is also considered in reinforcement learning, but limited work exists on comparing the two approaches on the same discrete-state environments. In this letter, we provide (1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in reinforcement learning, and (2) an explicit discrete-state comparison between active inference and reinforcement learning on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of reinforcement learning. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration-and account for uncertainty about their environment-in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in reinforcement learning is removed in active inference, where reward can simply be treated as another observation we have a preference over; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based reinforcement learning agents and by placing zero prior preferences over rewards and learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings (e.g., robotic arm movement, Atari games) if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation and demonstrate these behaviors in a OpenAI gym environment, alongside reinforcement learning agents

Optical Space Switches in Data Centers: Issues with Transport Protocols

A number of new architectures for data centre networks employing reconfigurable, SDN controlled, all-optical networks have been reported in recent years. In most cases, additional capacity was added to the system which unsurprisingly improved performance. In this study, a generalised network model that emulates the behaviour of these types of network was developed but where the total capacity is maintained constant so that system behaviour can be understood. An extensive emulated study is presented which indicates that the reconfiguration of such a network can have a detrimental impact on Transmission Control Protocol (TCP) congestion control mechanisms that can degrade the performance of the system. A number of simple scheduling mechanisms were investigated and the results show that an on-demand scheduling mechanism could deliver a throughput increase of more than ∼50% without any increase in total installed network capacity. These results, therefore, indicate the need to link the network resource management with new datacentre network architectures

Tackling the Tibetan Plateau in a down suit: Insights into thermoregulation by bar-headed geese during migration

This is the final version. Available from Company of Biologists via the DOI in this recordData accessibility: Following the manuscript being accepted data will be uploaded to a public repository such as Dryad.Birds migrating through extreme environments can experience a range of challenges while matching the demands of flight, including highly variable ambient temperatures, humidity and oxygen levels. However, there has been limited research into avian thermoregulation during migration in extreme environments. This study aimed to investigate the effect of flight performance and high-altitude on body temperature (Tb) of free flying bar-headed geese (Anser indicus), a species that completes a high-altitude trans-Himalayan migration through very cold, hypoxic environments. We measured abdominal Tb, along with altitude (via changes in barometric pressure), heart rate and body acceleration of bar-headed geese during their migration across the Tibetan Plateau. Bar-headed geese vary the circadian rhythm of Tb in response to migration, with peak daily Tb during daytime hours outside of migration but early in the morning or overnight during migration, reflecting changes in body acceleration. However, during flights changes in Tb were not consistent with changes in flight performance (as measured by heart rate or rate of ascent) or altitude. Overall, our results suggest that bar-headed geese are able to thermoregulate during high-altitude migration, maintaining Tb within a relatively narrow range despite appreciable variation in flight intensity and environmental conditions.Biotechnology and Biological Sciences Research Council (BBSRC)Natural Sciences and Engineering Research Council of Canada (NSERC)Max Planck Institute for OrnithologyUS Geological SurveyWestern Ecological and Patuxent Wildlife Research Centers, Avian Influenza Programm

Do Bar-Headed Geese Train for High Altitude Flights?

This is the author accepted manuscript. The final version is available from OUP via the DOI in this recordSYNOPSIS: Exercise at high altitude is extremely challenging, largely due to hypobaric hypoxia (low oxygen levels brought about by low air pressure). In humans, the maximal rate of oxygen consumption decreases with increasing altitude, supporting progressively poorer performance. Bar-headed geese (Anser indicus) are renowned high altitude migrants and, although they appear to minimize altitude during migration where possible, they must fly over the Tibetan Plateau (mean altitude 4800 m) for much of their annual migration. This requires considerable cardiovascular effort, but no study has assessed the extent to which bar-headed geese may train prior to migration for long distances, or for high altitudes. Using implanted loggers that recorded heart rate, acceleration, pressure, and temperature, we found no evidence of training for migration in bar-headed geese. Geese showed no significant change in summed activity per day or maximal activity per day. There was also no significant change in maximum heart rate per day or minimum resting heart rate, which may be evidence of an increase in cardiac stroke volume if all other variables were to remain the same. We discuss the strategies used by bar-headed geese in the context of training undertaken by human mountaineers when preparing for high altitude, noting the differences between their respective cardiovascular physiology.This work was supported by the UK Biotechnology and Biological Sciences Research Council [BBSRC; BB/FO15615/1 to C.M.B. and P.J.B.]. Authors were supported by a Natural Sciences and Engineering Research Council of Canada (NSERC) award [W.K.M.], and the FAO through the Animal Health Service EMPRES surveillance program

EquiFACS: the Equine Facial Action Coding System

Although previous studies of horses have investigated their facial expressions in specific contexts, e.g. pain, until now there has been no methodology available that documents all the possible facial movements of the horse and provides a way to record all potential facial configurations. This is essential for an objective description of horse facial expressions across a range of contexts that reflect different emotional states. Facial Action Coding Systems (FACS) provide a systematic methodology of identifying and coding facial expressions on the basis of underlying facial musculature and muscle movement. FACS are anatomically based and document all possible facial movements rather than a configuration of movements associated with a particular situation. Consequently, FACS can be applied as a tool for a wide range of research questions. We developed FACS for the domestic horse (Equus caballus) through anatomical investigation of the underlying musculature and subsequent analysis of naturally occurring behaviour captured on high quality video. Discrete facial movements were identified and described in terms of the underlying muscle contractions, in correspondence with previous FACS systems. The reliability of others to be able to learn this system (EquiFACS) and consistently code behavioural sequences was high—and this included people with no previous experience of horses. A wide range of facial movements were identified, including many that are also seen in primates and other domestic animals (dogs and cats). EquiFACS provides a method that can now be used to document the facial movements associated with different social contexts and thus to address questions relevant to understanding social cognition and comparative psychology, as well as informing current veterinary and animal welfare practices

Thalamocortical dynamics underlying spontaneous transitions in beta power in Parkinsonism

Parkinson's disease (PD) is a neurodegenerative condition in which aberrant oscillatory synchronization of neuronal activity at beta frequencies (15-35 Hz) across the cortico-basal ganglia-thalamocortical circuit is associated with debilitating motor symptoms, such as bradykinesia and rigidity. Mounting evidence suggests that the magnitude of beta synchrony in the parkinsonian state fluctuates over time, but the mechanisms by which thalamocortical circuitry regulates the dynamic properties of cortical beta in PD are poorly understood. Using the recently developed generic Dynamic Causal Modelling (DCM) framework, we recursively optimized a set of plausible models of the thalamocortical circuit (n = 144) to infer the neural mechanisms that best explain the transitions between low and high beta power states observed in recordings of field potentials made in the motor cortex of anesthetized Parkinsonian rats. Bayesian model comparison suggests that upregulation of cortical rhythmic activity in the beta-frequency band results from changes in the coupling strength both between and within the thalamus and motor cortex. Specifically, our model indicates that high levels of cortical beta synchrony are mainly achieved by a delayed (extrinsic) input from thalamic relay cells to deep pyramidal cells and a fast (intrinsic) input from middle pyramidal cells to superficial pyramidal cells. From a clinical perspective, our study provides insights into potential therapeutic strategies that could be utilized to modulate the network mechanisms responsible for the enhancement of cortical beta in PD. Specifically, we speculate that cortical stimulation aimed to reduce the enhanced excitatory inputs to either the superficial or deep pyramidal cells could be a potential non-invasive therapeutic strategy for PD

Experimental and theoretical investigation of ligand effects on the synthesis of ZnO nanoparticles

ZnO nanoparticles with highly controllable particle sizes(less than 10 nm) were synthesized using organic capping ligands in Zn(Ac)2 ethanolic solution. The molecular structure of the ligands was found to have significant influence on the particle size. The multi-functional molecule tris(hydroxymethyl)-aminomethane (THMA) favoured smaller particle distributions compared with ligands possessing long hydrocarbon chains that are more frequently employed. The adsorption of capping ligands on ZnnOn crystal nuclei (where n = 4 or 18 molecular clusters of(0001) ZnO surfaces) was modelled by ab initio methods at the density functional theory (DFT) level. For the molecules examined, chemisorption proceeded via the formation of Zn...O, Zn...N, or Zn...S chemical bonds between the ligands and active Zn2+ sites on ZnO surfaces. The DFT results indicated that THMA binds more strongly to the ZnO surface than other ligands, suggesting that this molecule is very effective at stabilizing ZnO nanoparticle surfaces. This study, therefore, provides new insight into the correlation between the molecular structure of capping ligands and the morphology of metal oxide nanostructures formed in their presence

Threat-sensitive anti-predator defence in precocial wader, the northern lapwing Vanellus vanellus

Birds exhibit various forms of anti-predator behaviours to avoid reproductive failure, with mobbing—observation, approach and usually harassment of a predator—being one of the most commonly observed. Here, we investigate patterns of temporal variation in the mobbing response exhibited by a precocial species, the northern lapwing (Vanellus vanellus). We test whether brood age and self-reliance, or the perceived risk posed by various predators, affect mobbing response of lapwings. We quantified aggressive interactions between lapwings and their natural avian predators and used generalized additive models to test how timing and predator species identity are related to the mobbing response of lapwings. Lapwings diversified mobbing response within the breeding season and depending on predator species. Raven Corvus corax, hooded crow Corvus cornix and harriers evoked the strongest response, while common buzzard Buteo buteo, white stork Ciconia ciconia, black-headed gull Chroicocephalus ridibundus and rook Corvus frugilegus were less frequently attacked. Lapwings increased their mobbing response against raven, common buzzard, white stork and rook throughout the breeding season, while defence against hooded crow, harriers and black-headed gull did not exhibit clear temporal patterns. Mobbing behaviour of lapwings apparently constitutes a flexible anti-predator strategy. The anti-predator response depends on predator species, which may suggest that lapwings distinguish between predator types and match mobbing response to the perceived hazard at different stages of the breeding cycle. We conclude that a single species may exhibit various patterns of temporal variation in anti-predator defence, which may correspond with various hypotheses derived from parental investment theory

Dogs and humans respond to emotionally competent stimuli by producing different facial actions

The commonality of facial expressions of emotion has been studied in different species since Darwin, with most of the research focusing on closely related primate species. However, it is unclear to what extent there exists common facial expression in species more phylogenetically distant, but sharing a need for common interspecific emotional understanding. Here we used the objective, anatomically-based tools, FACS and DogFACS (Facial Action Coding Systems), to quantify and compare human and domestic dog facial expressions in response to emotionally-competent stimuli associated with different categories of emotional arousal. We sought to answer two questions: Firstly, do dogs display specific discriminatory facial movements in response to different categories of emotional stimuli? Secondly, do dogs display similar facial movements to humans when reacting in emotionally comparable contexts? We found that dogs displayed distinctive facial actions depending on the category of stimuli. However, dogs produced different facial movements to humans in comparable states of emotional arousal. These results refute the commonality of emotional expression across mammals, since dogs do not display human-like facial expressions. Given the unique interspecific relationship between dogs and humans, two highly social but evolutionarily distant species sharing a common environment, these findings give new insight into the origin of emotion expression