Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses

Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from many random initial points. One thread of work has focused on explaining this phenomenon by characterizing the local curvature near critical points of the loss function, where the gradients are near zero, and demonstrating that neural network losses enjoy a no-bad-local-minima property and an abundance of saddle points. We report here that the methods used to find these putative critical points suffer from a bad local minima problem of their own: they often converge to or pass through regions where the gradient norm has a stationary point. We call these gradient-flat regions, since they arise when the gradient is approximately in the kernel of the Hessian, such that the loss is locally approximately linear, or flat, in the direction of the gradient. We describe how the presence of these regions necessitates care in both interpreting past results that claimed to find critical points of neural network losses and in designing second-order methods for optimizing neural networks.Comment: 18 pages, 5 figure

Corneal perforation in ocular graft-versus-host disease

PURPOSE: Corneal perforation is a rare, vision-threatening complication of ocular graft-versus-host disease (GVHD) and is not well understood. Our objective was to examine the clinical disease course and histopathologic correlation in patients who progressed to this outcome. METHODS: This study is a retrospective case series from four academic centers in the United States. All patients received a hematopoietic stem cell transplant (HSCT) prior to developing ocular GVHD. Variables of interest included patient demographics, time interval between HSCT and ocular events, visual acuity throughout clinical course, corticosteroid and infection prophylaxis regimens at time of corneal perforation, medical/surgical interventions, and histopathology. RESULTS: Fourteen eyes from 14 patients were analyzed. Most patients were male (86%) and Caucasian (86%), and average age at time of hematopoietic stem cell transplant was 47 years. The mean interval between hematopoietic stem cell transplant and diagnosis of ocular graft-versus-host disease was 9.5 months, and between hematopoietic stem cell transplant and corneal perforation was 37 months. Initial best-corrected visual acuity was 20/40 or better in 9 eyes, and all eyes had moderate or poor visual outcomes despite aggressive management, including corneal gluing in all patients followed by keratoplasty in 8 patients. The mean follow-up after perforation was 34 months (range 2-140 months). Oral prednisone was used prior to perforation in 11 patients (79%). On histopathology, representative specimens in the acute phase demonstrated ulcerative keratitis with perforation but minimal inflammatory cells and no microorganisms, consistent with sterile corneal melt in the setting of immunosuppression; and in the healed phase, filling in of the perforation site with fibrous scar. CONCLUSIONS: In these patients, an extended time interval was identified between the diagnosis of ocular graft-versus-host disease and corneal perforation. This represents a critical window to potentially prevent this devastating outcome. Further study is required to identify those patients at greatest risk as well as to optimize prevention strategies

The Cabibbo angle as a universal seed for quark and lepton mixings

A model-independent ansatz to describe lepton and quark mixing in a unified way is suggested based upon the Cabibbo angle. In our framework neutrinos mix in a ¿Bi-Large¿ fashion, while the charged leptons mix as the ¿down-type¿ quarks do. In addition to the standard Wolfenstein parameters (lambda, A) two other free parameters (psi, delta) are needed to specify the physical lepton mixing matrix. Through this simple assumption one makes specific predictions for the atmospheric angle as well as leptonic CP violation in good agreement with current observations

A novel Netrin-1-sensitive mechanism promotes local SNARE-mediated exocytosis during axon branching

Developmental axon branching dramatically increases synaptic capacity and neuronal surface area. Netrin-1 promotes branching and synaptogenesis, but the mechanism by which Netrin-1 stimulates plasma membrane expansion is unknown. We demonstrate that SNARE-mediated exocytosis is a prerequisite for axon branching and identify the E3 ubiquitin ligase TRIM9 as a critical catalytic link between Netrin-1 and exocytic SNARE machinery in murine cortical neurons. TRIM9 ligase activity promotes SNARE-mediated vesicle fusion and axon branching in a Netrin-dependent manner. We identified a direct interaction between TRIM9 and the Netrin-1 receptor DCC as well as a Netrin-1–sensitive interaction between TRIM9 and the SNARE component SNAP25. The interaction with SNAP25 negatively regulates SNARE-mediated exocytosis and axon branching in the absence of Netrin-1. Deletion of TRIM9 elevated exocytosis in vitro and increased axon branching in vitro and in vivo. Our data provide a novel model for the spatial regulation of axon branching by Netrin-1, in which localized plasma membrane expansion occurs via TRIM9-dependent regulation of SNARE-mediated vesicle fusion.American Heart Association (Fellowship 0615692T)National Institutes of Health (U.S.) (Grant GM68678

Evidence of maternal QTL affecting growth and obesity in adult mice

Most quantitative trait loci (QTL) studies fail to account for the effect that the maternal genotype may have on an individual’s phenotypes, even though maternal effect QTL have been shown to account for considerable variation in growth and obesity traits in mouse models. Moreover, the fetal programming theory suggests that maternal effects influence an offspring’s adult fitness, although the genetic nature of fetal programming remains unclear. Within this context, our study focused on mapping genomic regions associated with maternal effect QTL by analyzing the phenotypes of chromosomes 2 and 7 subcongenic mice from genetically distinct dams. We analyzed 12 chromosome 2 subcongenic strains that spanned from 70 to 180 Mb with CAST/EiJ donor regions on the background of C57BL/6 J, and 14 chromosome 7 subcongenic strains that spanned from 81 to 111 Mb with BALB/cByJ donor regions on C57BL/6ByJ background. Maternal QTL analyses were performed on the basis of overlapping donor regions between subcongenic strains. We identified several highly significant (P < 5 × 10−4) maternal QTL influencing total body weight, organ weight, and fat pad weights in both sets of subcongenics. These QTL accounted for 1.9-11.7% of the phenotypic variance for growth and obesity and greatly narrowed the genomic regions associated with the maternal genetic effects. These maternal effect QTL controlled phenotypic traits in adult mice, suggesting that maternal influences at early stages of development may permanently affect offspring performance. Identification of maternal effects in our survey of two sets of subcongenic strains, representing approximately 5% of the mouse genome, supports the hypothesis that maternal effects represent significant sources of genetic variation that are largely ignored in genetic studies

Essential Nutrients for Bone Health and a Review of their Availability in the Average North American Diet

Osteoporosis and low bone mineral density affect millions of Americans. The majority of adults in North America have insufficient intake of vitamin D and calcium along with inadequate exercise. Physicians are aware that vitamin D, calcium and exercise are essential for maintenance of bone health. Physicians are less likely to be aware that dietary insufficiencies of magnesium, silicon, Vitamin K, and boron are also widely prevalent, and each of these essential nutrients is an important contributor to bone health. In addition, specific nutritional factors may improve calcium metabolism and bone formation. It is the authors’ opinion that nutritional supplements should attempt to provide ample, but not excessive, amounts of factors that are frequently insufficient in the typical American diet

Mutations in the Mitochondrial Methionyl-tRNA Synthetase Cause a Neurodegenerative Phenotype in Flies and a Recessive Ataxia (ARSAL) in Humans

The study of Drosophila neurodegenerative mutants combined with genetic and biochemical analyses lead to the identification of multiple complex mutations in 60 patients with a novel form of ataxia/leukoencephalopathy