Continual Learning in Recurrent Neural Networks with Hypernetworks

The last decade has seen a surge of interest in continual learning (CL), and a variety of methods have been developed to alleviate catastrophic forgetting. However, most prior work has focused on tasks with static data, while CL on sequential data has remained largely unexplored. Here we address this gap in two ways. First, we evaluate the performance of established CL methods when applied to recurrent neural networks (RNNs). We primarily focus on elastic weight consolidation, which is limited by a stability-plasticity trade-off, and explore the particularities of this trade-off when using sequential data. We show that high working memory requirements, but not necessarily sequence length, lead to an increased need for stability at the cost of decreased performance on subsequent tasks. Second, to overcome this limitation we employ a recent method based on hypernetworks and apply it to RNNs to address catastrophic forgetting on sequential data. By generating the weights of a main RNN in a task-dependent manner, our approach disentangles stability and plasticity, and outperforms alternative methods in a range of experiments. Overall, our work provides several key insights on the differences between CL in feedforward networks and in RNNs, while offering a novel solution to effectively tackle CL on sequential data.Comment: 13 pages and 4 figures in the main text; 20 pages and 2 figures in the supplementary material

The first molecular phylogeny of Buthidae (Scorpiones)

The first partial phylogeny of family Buthidae (17 genera) is presented, based on molecular data (16S rRNA mitochondrial DNA). The strong support for a monophyletic Old World group of 13 genera (mainly Palearctic desert forms) is demonstrated, while representative genera from Madagascar (Grosphus) and Southeast Asia (Lychas) group outside, as well as New World genera Centruroides and Rhopalurus. A very strong support is observed for the first time for three groups of Old World genera: (a) Compsobuthus, Mesobuthus, Liobuthus, Kraepelinia; (b) Hottentotta, Buthacus; (c) Orthochirus, Anomalobuthus. Phylogenetic hypotheses are discussed

Paracrine IL-2 Is Required for Optimal Type 2 Effector Cytokine Production

IL-2 is a pleiotropic cytokine that promotes the differentiation of Th cell subsets, including Th1, Th2, and Th9 cells, but it impairs the development of Th17 and T follicular helper cells. Although IL-2 is produced by all polarized Th subsets to some level, how it impacts cytokine production when effector T cells are restimulated is unknown. We show in this article that Golgi transport inhibitors (GTIs) blocked IL-9 production. Mechanistically, GTIs blocked secretion of IL-2 that normally feeds back in a paracrine manner to promote STAT5 activation and IL-9 production. IL-2 feedback had no effect on Th1- or Th17-signature cytokine production, but it promoted Th2- and Th9-associated cytokine expression. These data suggest that the use of GTIs results in an underestimation of the presence of type 2 cytokine-secreting cells and highlight IL-2 as a critical component in optimal cytokine production by Th2 and Th9 cells in vitro and in vivo

New connections between finite element formulations of the Navier--Stokes equations

We show the velocity solutions to the convective, skew-symmetric, and rotational Galerkin finite element formulations of the Navier-Stokes equations are identical if Scott-Vogelius elements are used, and thus all three formulations will the same pointwise divergence free solution velocity. A connection is then established between the formulations for grad-div stabilized Taylor-Hood elements: under mild restrictions, the formulations' velocity solutions converge to each other (and to the Scott-Vogelius solution) as the stabilization parameter tends to infinity. Thus the benefits of using Scott-Vogelius elements can be obtained with the less expensive Taylor-Hood elements, and moreover the benefits of all the formulations can be retained if the rotational formulation is used. Numerical examples are provided that confirm the theory

Auto- and cross-power spectral analysis of dual trap optical tweezer experiments using Bayesian inference

The thermal fluctuations of micron-sized beads in dual trap optical tweezer experiments contain complete dynamic information about the viscoelastic properties of the embedding medium and—if present—macromolecular constructs connecting the two beads. To quantitatively interpret the spectral properties of the measured signals, a detailed understanding of the instrumental characteristics is required. To this end, we present a theoretical description of the signal processing in a typical dual trap optical tweezer experiment accounting for polarization crosstalk and instrumental noise and discuss the effect of finite statistics. To infer the unknown parameters from experimental data, a maximum likelihood method based on the statistical properties of the stochastic signals is derived. In a first step, the method can be used for calibration purposes: We propose a scheme involving three consecutive measurements (both traps empty, first one occupied and second empty, and vice versa), by which all instrumental and physical parameters of the setup are determined. We test our approach for a simple model system, namely a pair of unconnected, but hydrodynamically interacting spheres. The comparison to theoretical predictions based on instantaneous as well as retarded hydrodynamics emphasizes the importance of hydrodynamic retardation effects due to vorticity diffusion in the fluid. For more complex experimental scenarios, where macromolecular constructs are tethered between the two beads, the same maximum likelihood method in conjunction with dynamic deconvolution theory will in a second step allow one to determine the viscoelastic properties of the tethered element connecting the two beads

Autosomal Monoallelic Expression in the Mouse

Background: Random monoallelic expression defines an unusual class of genes displaying random choice for expression between the maternal and paternal alleles. Once established, the allele-specific expression pattern is stably maintained and mitotically inherited. Examples of random monoallelic genes include those found on the X-chromosome and a subset of autosomal genes, which have been most extensively studied in humans. Here, we report a genome-wide analysis of random monoallelic expression in the mouse. We used high density mouse genome polymorphism mapping arrays to assess allele-specific expression in clonal cell lines derived from heterozygous mouse strains. Results: Over 1,300 autosomal genes were assessed for allele-specific expression, and greater than 10% of them showed random monoallelic expression. When comparing mouse and human, the number of autosomal orthologs demonstrating random monoallelic expression in both organisms was greater than would be expected by chance. Random monoallelic expression on the mouse autosomes is broadly similar to that in human cells: it is widespread throughout the genome, lacks chromosome-wide coordination, and varies between cell types. However, for some mouse genes, there appears to be skewing, in some ways resembling skewed X-inactivation, wherein one allele is more frequently active. Conclusions: These data suggest that autosomal random monoallelic expression was present at least as far back as the last common ancestor of rodents and primates. Random monoallelic expression can lead to phenotypic variation beyond the phenotypic variation dictated by genotypic variation. Thus, it is important to take into account random monoallelic expression when examining genotype-phenotype correlation

Vector-Like Top Quark Production via an Electroweak Dipole Moment at a Muon Collider

Vectorial partners of the Standard Model quarks and leptons are predicted in many dynamical models of electroweak symmetry breaking. The most easily accessible of these new particles, either due to mass or couplings, are typically expected to be the partners of the third-generation fermions. It is therefore essential to explore the signatures of these particles at future high-energy colliders. We study the potential of a high-energy muon collider to singly produce a vector-like top-quark partner via an electroweak dipole moment operator, such an operator being typical of composite constructions beyond the Standard Model. We use a phenomenological model for third-generation quarks and their partners that satisfies an extended custodial symmetry. This automatically protects the $W$-boson and $Z$-boson masses from receiving large electroweak corrections, and it allows the model to be viable given current electroweak data. We demonstrate that cross sections associated with dipole-induced vector-like quark production can easily exceed those inherent to more conventional single-production modes via ordinary electroweak couplings. We then explore the associated phenomenology, and we show that at least one (and often more than one) of the extra vector-like states can be studied at high-energy muon colliders. Typical accessible masses are found to range up to close to the kinematic production threshold, when the vector-like partners are produced in combination with an ordinary top quark.Comment: 37 pages, 11 figures, 1 table. matches published versio

Teraflop per second gravitational lensing ray-shooting using graphics processing units

Gravitational lensing calculation using a direct inverse ray-shooting approach is a computationally expensive way to determine magnification maps, caustic patterns, and light-curves (e.g. as a function of source profile and size). However, as an easily parallelisable calculation, gravitational ray-shooting can be accelerated using programmable graphics processing units (GPUs). We present our implementation of inverse ray-shooting for the NVIDIA G80 generation of graphics processors using the NVIDIA Compute Unified Device Architecture (CUDA) software development kit. We also extend our code to multiple-GPU systems, including a 4-GPU NVIDIA S1070 Tesla unit. We achieve sustained processing performance of 182 Gflop/s on a single GPU, and 1.28 Tflop/s using the Tesla unit. We demonstrate that billion-lens microlensing simulations can be run on a single computer with a Tesla unit in timescales of order a day without the use of a hierarchical tree code.Comment: 21 pages, 4 figures, submitted to New Astronom

The first molecular phylogeny of Buthidae (Scorpiones)

The first partial phylogeny of family Buthidae (17 genera) is presented, based on molecular data (16S rRNA mitochondrial DNA). The strong support for a monophyletic Old World group of 13 genera (mainly Palearctic desert forms) is demonstrated, while representative genera from Madagascar (Grosphus) and Southeast Asia (Lychas) group outside, as well as New World genera Centruroides and Rhopalurus. A very strong support is observed for the first time for three groups of Old World genera: (a) Compsobuthus, Mesobuthus, Liobuthus, Kraepelinia; (b) Hottentotta, Buthacus; (c) Orthochirus, Anomalobuthus. Phylogenetic hypotheses are discussed