Bifurcations and loss jumps in RNN training

Recurrent neural networks (RNNs) are popular machine learning tools for modeling and forecasting sequential data and for inferring dynamical systems (DS) from observed time series. Concepts from DS theory (DST) have variously been used to further our understanding of both, how trained RNNs solve complex tasks, and the training process itself. Bifurcations are particularly important phenomena in DS, including RNNs, that refer to topological (qualitative) changes in a system's dynamical behavior as one or more of its parameters are varied. Knowing the bifurcation structure of an RNN will thus allow to deduce many of its computational and dynamical properties, like its sensitivity to parameter variations or its behavior during training. In particular, bifurcations may account for sudden loss jumps observed in RNN training that could severely impede the training process. Here we first mathematically prove for a particular class of ReLU-based RNNs that certain bifurcations are indeed associated with loss gradients tending toward infinity or zero. We then introduce a novel heuristic algorithm for detecting all fixed points and k-cycles in ReLU-based RNNs and their existence and stability regions, hence bifurcation manifolds in parameter space. In contrast to previous numerical algorithms for finding fixed points and common continuation methods, our algorithm provides exact results and returns fixed points and cycles up to high orders with surprisingly good scaling behavior. We exemplify the algorithm on the analysis of the training process of RNNs, and find that the recently introduced technique of generalized teacher forcing completely avoids certain types of bifurcations in training. Thus, besides facilitating the DST analysis of trained RNNs, our algorithm provides a powerful instrument for analyzing the training process itself

Urinary NMR Profiling in Pediatric Acute Kidney Injury—A Pilot Study

Acute kidney injury (AKI) in critically ill children and adults is associated with significant short- and long-term morbidity and mortality. As serum creatinine- and urine output-based definitions of AKI have relevant limitations, there is a persistent need for better diagnostics of AKI. Nuclear magnetic resonance (NMR) spectroscopy allows for analysis of metabolic profiles without extensive sample manipulations. In the study reported here, we examined the diagnostic accuracy of NMR urine metabolite patterns for the diagnosis of neonatal and pediatric AKI according to the Kidney Disease: Improving Global Outcomes (KDIGO) definition. A cohort of 65 neonatal and pediatric patients (0–18 years) with established AKI of heterogeneous etiology was compared to both a group of apparently healthy children (n = 53) and a group of critically ill children without AKI (n = 31). Multivariate analysis identified a panel of four metabolites that allowed diagnosis of AKI with an area under the receiver operating characteristics curve (AUC-ROC) of 0.95 (95% confidence interval 0.86–1.00). Especially urinary citrate levels were significantly reduced whereas leucine and valine levels were elevated. Metabolomic differentiation of AKI causes appeared promising but these results need to be validated in larger studies. In conclusion, this study shows that NMR spectroscopy yields high diagnostic accuracy for AKI in pediatric patients

Orientational properties of nematic disclinations

Topological defects play a pivotal role in the physics of liquid crystals and represent one of the most prominent and well studied aspects of mesophases. While in two-dimensional nematics, disclinations are traditionally treated as point-like objects, recent experimental studies on active nematics have suggested that half-strength disclinations might in fact possess a polar structure. In this article, we provide a precise definition of polarity for half-strength nematic disclinations, we introduce a simple and robust method to calculate this quantity from experimental and numerical data and we investigate how the orientational properties of half-strength disclinations affect their relaxational dynamics.Comment: 6 pages, 5 figures, supplementary movies at http://wwwhome.lorentz.leidenuniv.nl/~giomi/sup_mat/20150720

Aerobic capacity, activity levels and daily energy expenditure in male and female adolescents of the kenyan nandi sub-group

The relative importance of genetic and socio-cultural influences contributing to the success of east Africans in endurance athletics remains unknown in part because the pre-training phenotype of this population remains incompletely assessed. Here cardiopulmonary fitness, physical activity levels, distance travelled to school and daily energy expenditure in 15 habitually active male (13.9±1.6 years) and 15 habitually active female (13.9±1.2) adolescents from a rural Nandi primary school are assessed. Aerobic capacity ([Formula: see text]) was evaluated during two maximal discontinuous incremental exercise tests; physical activity using accelerometry combined with a global positioning system; and energy expenditure using the doubly labelled water method. The [Formula: see text] of the male and female adolescents were 73.9±5.7 ml(.) kg(-1.) min(-1) and 61.5±6.3 ml(.) kg(-1.) min(-1), respectively. Total time spent in sedentary, light, moderate and vigorous physical activities per day was 406±63 min (50% of total monitored time), 244±56 min (30%), 75±18 min (9%) and 82±30 min (10%). Average total daily distance travelled to and from school was 7.5±3.0 km (0.8-13.4 km). Mean daily energy expenditure, activity-induced energy expenditure and physical activity level was 12.2±3.4 MJ(.) day(-1), 5.4±3.0 MJ(.) day(-1) and 2.2±0.6. 70.6% of the variation in [Formula: see text] was explained by sex (partial R(2) = 54.7%) and body mass index (partial R(2) = 15.9%). Energy expenditure and physical activity variables did not predict variation in [Formula: see text] once sex had been accounted for. The highly active and energy-demanding lifestyle of rural Kenyan adolescents may account for their exceptional aerobic fitness and collectively prime them for later training and athletic success

Inactivation of thermogenic UCP1 as a historical contingency in multiple placental mammal clades

Mitochondrial uncoupling protein 1 (UCP1) is essential for nonshivering thermogenesis in brown adipose tissue and is widely accepted to have played a key thermoregulatory role in small-bodied and neonatal placental mammals that enabled the exploitation of cold environments. We map ucp1 sequences from 133 mammals onto a species tree constructed from a ∼51-kb sequence alignment and show that inactivating mutations have occurred in at least 8 of the 18 traditional placental orders, thereby challenging the physiological importance of UCP1 across Placentalia. Selection and timetree analyses further reveal that ucp1 inactivations temporally correspond with strong secondary reductions in metabolic intensity in xenarthrans and pangolins, or in six other lineages coincided with a ∼30 million-year episode of global cooling in the Paleogene that promoted sharp increases in body mass and cladogenesis evident in the fossil record. Our findings also demonstrate that members of various lineages (for example, cetaceans, horses, woolly mammoths, Steller's sea cows) evolved extreme cold hardiness in the absence of UCP1-mediated thermogenesis. Finally, we identify ucp1 inactivation as a historical contingency that is linked to the current low species diversity of clades lacking functional UCP1, thus providing the first evidence for species selection related to the presence or absence of a single gene product

The Germinal Center Kinase GCK-1 Is a Negative Regulator of MAP Kinase Activation and Apoptosis in the C. elegans Germline

The germinal center kinases (GCK) constitute a large, highly conserved family of proteins that has been implicated in a wide variety of cellular processes including cell growth and proliferation, polarity, migration, and stress responses. Although diverse, these functions have been attributed to an evolutionarily conserved role for GCKs in the activation of ERK, JNK, and p38 MAP kinase pathways. In addition, multiple GCKs from different species promote apoptotic cell death. In contrast to these paradigms, we found that a C. elegans GCK, GCK-1, functions to inhibit MAP kinase activation and apoptosis in the C. elegans germline. In the absence of GCK-1, a specific MAP kinase isoform is ectopically activated and oocytes undergo abnormal development. Moreover, GCK-1- deficient animals display a significant increase in germ cell death. Our results suggest that individual germinal center kinases act in mechanistically distinct ways and that these functions are likely to depend on organ- and developmental-specific contexts

Ten millennia of hepatitis B virus evolution

Hepatitis B virus (HBV) has been infecting humans for millennia and remains a global health problem, but its past diversity and dispersal routes are largely unknown. We generated HBV genomic data from 137 Eurasians and Native Americans dated between ~10,500 and ~400 years ago. We date the most recent common ancestor of all HBV lineages to between ~20,000 and 12,000 years ago, with the virus present in European and South American hunter-gatherers during the early Holocene. After the European Neolithic transition, Mesolithic HBV strains were replaced by a lineage likely disseminated by early farmers that prevailed throughout western Eurasia for ~4000 years, declining around the end of the 2nd millennium BCE. The only remnant of this prehistoric HBV diversity is the rare genotype G, which appears to have reemerged during the HIV pandemic

Search for gravitational-lensing signatures in the full third observing run of the LIGO-Virgo network

Gravitational lensing by massive objects along the line of sight to the source causes distortions of gravitational wave-signals; such distortions may reveal information about fundamental physics, cosmology and astrophysics. In this work, we have extended the search for lensing signatures to all binary black hole events from the third observing run of the LIGO--Virgo network. We search for repeated signals from strong lensing by 1) performing targeted searches for subthreshold signals, 2) calculating the degree of overlap amongst the intrinsic parameters and sky location of pairs of signals, 3) comparing the similarities of the spectrograms amongst pairs of signals, and 4) performing dual-signal Bayesian analysis that takes into account selection effects and astrophysical knowledge. We also search for distortions to the gravitational waveform caused by 1) frequency-independent phase shifts in strongly lensed images, and 2) frequency-dependent modulation of the amplitude and phase due to point masses. None of these searches yields significant evidence for lensing. Finally, we use the non-detection of gravitational-wave lensing to constrain the lensing rate based on the latest merger-rate estimates and the fraction of dark matter composed of compact objects