Optimality-based Analysis of XCSF Compaction in Discrete Reinforcement Learning

Learning classifier systems (LCSs) are population-based predictive systems that were originally envisioned as agents to act in reinforcement learning (RL) environments. These systems can suffer from population bloat and so are amenable to compaction techniques that try to strike a balance between population size and performance. A well-studied LCS architecture is XCSF, which in the RL setting acts as a Q-function approximator. We apply XCSF to a deterministic and stochastic variant of the FrozenLake8x8 environment from OpenAI Gym, with its performance compared in terms of function approximation error and policy accuracy to the optimal Q-functions and policies produced by solving the environments via dynamic programming. We then introduce a novel compaction algorithm (Greedy Niche Mass Compaction - GNMC) and study its operation on XCSF's trained populations. Results show that given a suitable parametrisation, GNMC preserves or even slightly improves function approximation error while yielding a significant reduction in population size. Reasonable preservation of policy accuracy also occurs, and we link this metric to the commonly used steps-to-goal metric in maze-like environments, illustrating how the metrics are complementary rather than competitive

Multiplicity of fibronectin-binding alpha V integrin receptors in colorectal cancer.

Current data from in vitro and in vivo animal models indicate that fibronectin-binding integrin receptors expressed by colon cancer cells may regulate tumour growth. While individual members of the beta 1 subfamily of integrins have now been clearly identified in colorectal cancer, little information exists with respect to the alpha V subfamily. In the present study we show that alpha V can associate with multiple and different beta subunits capable of binding fibronectin in this tumour type. This is likely to have functional implications for growth and spread of colorectal cancer

A brief history of learning classifier systems: from CS-1 to XCS and its variants

© 2015, Springer-Verlag Berlin Heidelberg. The direction set by Wilson’s XCS is that modern Learning Classifier Systems can be characterized by their use of rule accuracy as the utility metric for the search algorithm(s) discovering useful rules. Such searching typically takes place within the restricted space of co-active rules for efficiency. This paper gives an overview of the evolution of Learning Classifier Systems up to XCS, and then of some of the subsequent developments of Wilson’s algorithm to different types of learning

Biotic carbon feedbacks in a materially-closed soil-vegetation-atmosphere system

The magnitude and direction of the coupled feedbacks between the biotic and abiotic components of the terrestrial carbon cycle is a major source of uncertainty in coupled climate–carbon-cycle models1, 2, 3. Materially closed, energetically open biological systems continuously and simultaneously allow the two-way feedback loop between the biotic and abiotic components to take place4, 5, 6, 7, but so far have not been used to their full potential in ecological research, owing to the challenge of achieving sustainable model systems6, 7. We show that using materially closed soil–vegetation–atmosphere systems with pro rata carbon amounts for the main terrestrial carbon pools enables the establishment of conditions that balance plant carbon assimilation, and autotrophic and heterotrophic respiration fluxes over periods suitable to investigate short-term biotic carbon feedbacks. Using this approach, we tested an alternative way of assessing the impact of increased CO2 and temperature on biotic carbon feedbacks. The results show that without nutrient and water limitations, the short-term biotic responses could potentially buffer a temperature increase of 2.3 °C without significant positive feedbacks to atmospheric CO2. We argue that such closed-system research represents an important test-bed platform for model validation and parameterization of plant and soil biotic responses to environmental changes

Pauli's Principle in Probe Microscopy

Exceptionally clear images of intramolecular structure can be attained in dynamic force microscopy through the combination of a passivated tip apex and operation in what has become known as the "Pauli exclusion regime" of the tip-sample interaction. We discuss, from an experimentalist's perspective, a number of aspects of the exclusion principle which underpin this ability to achieve submolecular resolution. Our particular focus is on the origins, history, and interpretation of Pauli's principle in the context of interatomic and intermolecular interactions.Comment: This is a chapter from "Imaging and Manipulation of Adsorbates using Dynamic Force Microscopy", a book which is part of the "Advances in Atom and Single Molecule Machines" series published by Springer [http://www.springer.com/series/10425]. To be published late 201

Self-reported neurological symptoms in relation to CO emissions due to problem gas appliance installations in London: a cross-sectional survey

Background: Previous research by the authors found evidence that up to 10% of particular household categories may be exposed to elevated carbon monoxide (CO) concentrations from poor quality gas appliance installations. The literature suggests certain neurological symptoms are linked to exposure to low levels of CO. This paper addresses the hypothesis that certain self-reported neurological symptoms experienced by a householder are linked to an estimate of their CO exposure.Methods: Between 27 April and 27 June 2006, 597 homes with a mains supply of natural gas were surveyed, mainly in old, urban areas of London. Qualified gas engineers tested all gas appliances (cooker, boiler, gas fire, and water heater) and reported, according to the Gas Industry Unsafe Situations Procedure, appliances considered At Risk (AR), Immediately Dangerous (ID) or Not to Current Standards (NCS). Five exposure risk categories were defined based on measurement of CO emitted by the appliance, its features and its use, with "high or very high" exposure category where occupants were considered likely to be exposed to levels greater than 26 ppm for one hour. The prevalence of symptoms at each level of exposure was compared with that at lowest level of exposure.Results: Of the households, 6% were assessed as having a "high or very high" risk of exposure to CO. Of the individuals, 9% reported at least one neurological symptom. There was a statistically significant association between "high or very high" exposure risk to CO and self-reported symptoms compared to "no exposure" likelihood, for households not in receipt of benefit, controlling for "number of residents" and presence of pensioners, OR = 3.23 (95% CI: 1.28, 8.15). Risk ratios across all categories of exposure likelihood indicate a dose-response pattern. Those households in receipt of benefit showed no dose-response pattern.Conclusion: This study found an association between risk of CO exposure at low concentration, and prevalence of self-reported neurological symptoms in the community for those households not in receipt of benefit. As health status was self-reported, this association requires further investigation

Topoisomerase II beta interacts with cohesin and CTCF at topological domain borders

BACKGROUND: Type II DNA topoisomerases (TOP2) regulate DNA topology by generating transient double stranded breaks during replication and transcription. Topoisomerase II beta (TOP2B) facilitates rapid gene expression and functions at the later stages of development and differentiation. To gain new insight into the genome biology of TOP2B, we used proteomics (BioID), chromatin immunoprecipitation, and high-throughput chromosome conformation capture (Hi-C) to identify novel proximal TOP2B protein interactions and characterize the genomic landscape of TOP2B binding at base pair resolution. RESULTS: Our human TOP2B proximal protein interaction network included members of the cohesin complex and nucleolar proteins associated with rDNA biology. TOP2B associates with DNase I hypersensitivity sites, allele-specific transcription factor (TF) binding, and evolutionarily conserved TF binding sites on the mouse genome. Approximately half of all CTCF/cohesion-bound regions coincided with TOP2B binding. Base pair resolution ChIP-exo mapping of TOP2B, CTCF, and cohesin sites revealed a striking structural ordering of these proteins along the genome relative to the CTCF motif. These ordered TOP2B-CTCF-cohesin sites flank the boundaries of topologically associating domains (TADs) with TOP2B positioned externally and cohesin internally to the domain loop. CONCLUSIONS: TOP2B is positioned to solve topological problems at diverse cis-regulatory elements and its occupancy is a highly ordered and prevalent feature of CTCF/cohesin binding sites that flank TADs

Bilateral Assessment of Functional Tasks for Robot-assisted Therapy Applications

This article presents a novel evaluation system along with methods to evaluate bilateral coordination of arm function on activities of daily living tasks before and after robot-assisted therapy. An affordable bilateral assessment system (BiAS) consisting of two mini-passive measuring units modeled as three degree of freedom robots is described. The process for evaluating functional tasks using the BiAS is presented and we demonstrate its ability to measure wrist kinematic trajectories. Three metrics, phase difference, movement overlap, and task completion time, are used to evaluate the BiAS system on a bilateral symmetric (bi-drink) and a bilateral asymmetric (bi-pour) functional task. Wrist position and velocity trajectories are evaluated using these metrics to provide insight into temporal and spatial bilateral deficits after stroke. The BiAS system quantified movements of the wrists during functional tasks and detected differences in impaired and unimpaired arm movements. Case studies showed that stroke patients compared to healthy subjects move slower and are less likely to use their arm simultaneously even when the functional task requires simultaneous movement. After robot-assisted therapy, interlimb coordination spatial deficits moved toward normal coordination on functional tasks