Pseudorehearsal in value function approximation

Catastrophic forgetting is of special importance in reinforcement learning, as the data distribution is generally non-stationary over time. We study and compare several pseudorehearsal approaches for Q-learning with function approximation in a pole balancing task. We have found that pseudorehearsal seems to assist learning even in such very simple problems, given proper initialization of the rehearsal parameters

Polarimetry and Spectroscopy of the `Oxygen Flaring' DQ Herculis-like nova: V5668 Sagittarii (2015)

Classical novae are eruptions on the surface of a white dwarf in a binary system. The material ejected from the white dwarf surface generally forms an axisymmetric shell of gas and dust around the system. The three-dimensional structure of these shells is difficult to untangle when viewed on the plane of the sky. In this work a geometrical model is developed to explain new observations of the 2015 nova V5668 Sagittarii. To understand the ionisation structure in terms of the nova shell morphology and estimate the emission distribution directly following the light-curve's dust-dip. High-cadence optical polarimetry and spectroscopy observations of a nova are presented. The ejecta is modelled in terms of morpho-kinematics and photoionisation structure. Initially observational results are presented, including broadband polarimetry and spectroscopy of V5668 Sgr nova during eruption. Variability over these observations provides clues towards the evolving structure of the nova shell. The position angle of the shell is derived from polarimetry, which is attributed to scattering from small dust grains. Shocks in the nova outflow are suggested in the photometry and the effect of these on the nova shell are illustrated with various physical diagnostics. Changes in density and temperature as the super soft source phase of the nova began are discussed. Gas densities are found to be of the order of 10$^{9}$ cm$^{-3}$ for the nova in its auroral phase. The blackbody temperature of the central stellar system is estimated to be around $2.2\times10^{5}$ K at times coincident with the super soft source turn-on. It was found that the blend around 4640 $\rm{\AA}$ commonly called `nitrogen flaring' is more naturally explained as flaring of the O~{\sc ii} multiplet (V1) from 4638 - 4696 $\rm{\AA}$, i.e. `oxygen flaring'

Assessing Neurofilaments as Biomarkers of Neuroprotection in Progressive Multiple Sclerosis: From the MS-STAT Randomized Controlled Trial

BACKGROUND AND OBJECTIVES: Improved biomarkers of neuroprotective treatment are needed in progressive multiple sclerosis (PMS) to facilitate more efficient phase 2 trial design. The MS-STAT randomized controlled trial supported the neuroprotective potential of high-dose simvastatin in secondary progressive MS (SPMS). Here, we analyze serum from the MS-STAT trial to assess the extent to which neurofilament light (NfL) and neurofilament heavy (NfH), both promising biomarkers of neuroaxonal injury, may act as biomarkers of simvastatin treatment in SPMS. METHODS: The MS-STAT trial randomized patients to 80 mg simvastatin or placebo. Serum was analyzed for NfL and NfH using Simoa technology. We used linear mixed models to investigate the treatment effects of simvastatin compared with placebo on NfL and NfH. Additional models examined the relationships between neurofilaments and MRI and clinical measures of disease severity. RESULTS: A total of 140 patients with SPMS were included. There was no evidence for a simvastatin treatment effect on NfL or NfH: compared with placebo, NfL was 1.2% lower (95% CI 10.6% lower to 9.2% higher; p = 0.820) and NfH was 0.4% lower (95% CI 18.4% lower to 21.6% higher; p = 0.969) in the simvastatin treatment group. Secondary analyses suggested that higher NfL was associated with greater subsequent whole brain atrophy, higher T2 lesion volume, and more new/enlarging T2 lesions in the previous 12 months, as well as greater physical disability. There were no significant associations between NfH and MRI or clinical variables. DISCUSSION: We found no evidence of a simvastatin treatment effect on serum neurofilaments. While confirmation of the neuroprotective benefits of simvastatin is awaited from the ongoing phase 3 study (NCT03387670), our results suggest that treatments capable of slowing the rate of whole brain atrophy in SPMS, such as simvastatin, may act via mechanisms largely independent of neuroaxonal injury, as quantified by NfL. This has important implications for the design of future phase 2 clinical trials in PMS. TRIAL REGISTRATION INFORMATION: MS-STAT: NCT00647348. CLASSIFICATION OF EVIDENCE: This study provides class I evidence that simvastatin treatment does not have a large impact on either serum NfL or NfH, as quantified in this study, in SPMS

Automated multi-objective calibration of biological agent-based simulations

Computational agent-based simulation (ABS) is increasingly used to complement laboratory techniques in advancing our understanding of biological systems. Calibration, the identification of parameter values that align simulation with biological behaviours, becomes challenging as increasingly complex biological domains are simulated. Complex domains cannot be characterized by single metrics alone, rendering simulation calibration a fundamentally multi-metric optimization problem that typical calibration techniques cannot handle. Yet calibration is an essential activity in simulation-based science; the baseline calibration forms a control for subsequent experimentation and hence is fundamental in the interpretation of results. Here, we develop and showcase a method, built around multi-objective optimization, for calibrating ABSs against complex target behaviours requiring several metrics (termed objectives) to characterize. Multi-objective calibration (MOC) delivers those sets of parameter values representing optimal trade-offs in simulation performance against each metric, in the form of a Pareto front. We use MOC to calibrate a well-understood immunological simulation against both established a priori and previously unestablished target behaviours. Furthermore, we show that simulation-borne conclusions are broadly, but not entirely, robust to adopting baseline parameter values from different extremes of the Pareto front, highlighting the importance of MOC's identification of numerous calibration solutions. We devise a method for detecting overfitting in a multi-objective context, not previously possible, used to save computational effort by terminating MOC when no improved solutions will be found. MOC can significantly impact biological simulation, adding rigour to and speeding up an otherwise time-consuming calibration process and highlighting inappropriate biological capture by simulations that cannot be well calibrated. As such, it produces more accurate simulations that generate more informative biological predictions

Gaussian Process Pseudo-Likelihood Models for Sequence Labeling

Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. The pseudo-likelihood model enables one to capture long range dependencies among the output components of the sequence without becoming computationally intractable. We use an efficient variational Gaussian approximation method to perform inference in the proposed model. We also provide an iterative algorithm which can effectively make use of the information from the neighboring labels to perform prediction. The ability to capture long range dependencies makes the proposed approach useful for a wide range of sequence labeling problems. Numerical experiments on some sequence labeling data sets demonstrate the usefulness of the proposed approach.Comment: 18 pages, 5 figure

Fossil evidence for key innovations in the evolution of insect diversity

Explaining the taxonomic richness of the insects, comprising over half of all described species, is a major challenge in evolutionary biology. Previously, several evolutionary novelties (key innovations) have been posited to contribute to that richness, including the insect bauplan, wings, wing folding and complete metamorphosis, but evidence over their relative importance and modes of action is sparse and equivocal. Here, a new dataset on the first and last occurrences of fossil hexapod (insects and close relatives) families is used to show that basal families of winged insects (Palaeoptera, e.g. dragonflies) show higher origination and extinction rates in the fossil record than basal wingless groups (Apterygota, e.g. silverfish). Origination and extinction rates were maintained at levels similar to Palaeoptera in the more derived Polyneoptera (e.g. cockroaches) and Paraneoptera (e.g. true bugs), but extinction rates subsequently reduced in the very rich group of insects with complete metamorphosis (Holometabola, e.g. beetles). Holometabola show evidence of a recent slow-down in their high net diversification rate, whereas other winged taxa continue to diversify at constant but low rates. These data suggest that wings and complete metamorphosis have had the most effect on family-level insect macroevolution, and point to specific mechanisms by which they have influenced insect diversity through time

Evolutionary stasis and lability in thermal physiology in a group of tropical lizards

Understanding how quickly physiological traits evolve is a topic of great interest, particularly in the context of how organisms can adapt in response to climate warming. Adjustment to novel thermal habitats may occur either through behavioural adjustments, physiological adaptation, or both. Here we test whether rates of evolution differ among physiological traits in the cybotoids, a clade of tropical Anolis lizards distributed in markedly different thermal environments on the Caribbean island of Hispaniola. We find that cold tolerance evolves considerably faster than heat tolerance, a difference that results because behavioural thermoregulation more effectively shields these organisms from selection on upper than lower temperature tolerances. Specifically, because lizards in very different environments behaviourally thermoregulate during the day to similar body temperatures, divergent selection on body temperature and heat tolerance is precluded, whereas night-time temperatures can only be partially buffered by behaviour, thereby exposing organisms to selection on cold tolerance. We discuss how exposure to selection on physiology influences divergence among tropical organisms and its implications for adaptive evolutionary response to climate warming

High loading of polygenic risk for ADHD in children with comorbid aggression

Objective: Although attention deficit hyperactivity disorder (ADHD) is highly heritable, genome-wide association studies (GWAS) have not yet identified any common genetic variants that contribute to risk. There is evidence that aggression or conduct disorder in children with ADHD indexes higher genetic loading and clinical severity. The authors examine whether common genetic variants considered en masse as polygenic scores for ADHD are especially enriched in children with comorbid conduct disorder. Method: Polygenic scores derived from an ADHD GWAS meta-analysis were calculated in an independent ADHD sample (452 case subjects, 5,081 comparison subjects). Multivariate logistic regression analyses were employed to compare polygenic scores in the ADHD and comparison groups and test for higher scores in ADHD case subjects with comorbid conduct disorder relative to comparison subjects and relative to those without comorbid conduct disorder. Association with symptom scores was tested using linear regression. Results: Polygenic risk for ADD, derived from the meta-analysis, was higher in the independent ADHD group than in the comparison group. Polygenic score was significantly higher in ADHD case subjects with conduct disorder relative to ADHD case subjects without conduct disorder. ADHD polygenic score showed significant association with comorbid conduct disorder symptoms. This relationship was explained by,the aggression items. Conclusions: Common genetic variation is relevant to ADHD, especially in individuals with comorbid aggression. The findings suggest that the previously published ADHD GWAS meta-analysis contains weak but true associations with common variants, support for which falls below genome-wide significance levels. The findings also highlight the fact that aggression in ADHD indexes genetic as well as clinical severity

Generation and quality control of lipidomics data for the alzheimers disease neuroimaging initiative cohort.

Alzheimers disease (AD) is a major public health priority with a large socioeconomic burden and complex etiology. The Alzheimer Disease Metabolomics Consortium (ADMC) and the Alzheimer Disease Neuroimaging Initiative (ADNI) aim to gain new biological insights in the disease etiology. We report here an untargeted lipidomics of serum specimens of 806 subjects within the ADNI1 cohort (188 AD, 392 mild cognitive impairment and 226 cognitively normal subjects) along with 83 quality control samples. Lipids were detected and measured using an ultra-high-performance liquid chromatography quadruple/time-of-flight mass spectrometry (UHPLC-QTOF MS) instrument operated in both negative and positive electrospray ionization modes. The dataset includes a total 513 unique lipid species out of which 341 are known lipids. For over 95% of the detected lipids, a relative standard deviation of better than 20% was achieved in the quality control samples, indicating high technical reproducibility. Association modeling of this dataset and available clinical, metabolomics and drug-use data will provide novel insights into the AD etiology. These datasets are available at the ADNI repository at http://adni.loni.usc.edu/

Recognition memory, self-other source memory, and theory-of-mind in children with autism spectrum disorder.

This study investigated semantic and episodic memory in autism spectrum disorder (ASD), using a task which assessed recognition and self-other source memory. Children with ASD showed undiminished recognition memory but significantly diminished source memory, relative to age- and verbal ability-matched comparison children. Both children with and without ASD showed an “enactment effect”, demonstrating significantly better recognition and source memory for self-performed actions than other-person-performed actions. Within the comparison group, theory-of-mind (ToM) task performance was significantly correlated with source memory, specifically for other-person-performed actions (after statistically controlling for verbal ability). Within the ASD group, ToM task performance was not significantly correlated with source memory (after controlling for verbal ability). Possible explanations for these relations between source memory and ToM are considered