Spectral pruning of fully connected layers

Training of neural networks can be reformulated in spectral space, by allowing eigenvalues and eigenvectors of the network to act as target of the optimization instead of the individual weights. Working in this setting, we show that the eigenvalues can be used to rank the nodes' importance within the ensemble. Indeed, we will prove that sorting the nodes based on their associated eigenvalues, enables effective pre- and post-processing pruning strategies to yield massively compacted networks (in terms of the number of composing neurons) with virtually unchanged performance. The proposed methods are tested for different architectures, with just a single or multiple hidden layers, and against distinct classification tasks of general interest.Comment: 16 pages, 11 figures. Sections rearranged in v

Sex Differences in the Response to Viral Infections: TLR8 and TLR9 Ligand Stimulation Induce Higher IL10 Production in Males

BACKGROUND: Susceptibility to viral infections as well as their severity are higher in men than in women. Heightened antiviral responses typical of women are effective for rapid virus clearance, but if excessively high or prolonged, can result in chronic/inflammatory pathologies. We investigated whether this variability could be in part attributable to differences in the response to the Toll-Like Receptors (TLR) more involved in the virus recognition. METHODS: Cytokine production by peripheral blood mononuclear cells (PBMCs) from male and female healthy donors after stimulation with Toll-like receptors (TLR) 3, 7, 8, 9 ligands or with viruses (influenza and Herpes-simplex-1) was evaluated. RESULTS: Compared to females, PBMCs from males produced not only lower amounts of IFN-α in response to TLR7 ligands but also higher amounts of the immunosuppressive cytokine IL10 after stimulation with TLR8 and TLR9 ligands or viruses. IL10 production after TLR9 ligands or HSV-1 stimulation was significantly related with plasma levels of sex hormones in both groups, whereas no correlation was found in cytokines produced following TLR7 and TLR8 stimulation. CONCLUSIONS: Given the role of an early production of IL10 by cells of innate immunity in modulating innate and adaptive immune response to viruses, we suggest that sex-related difference in its production following viral nucleic acid stimulation of TLRs may be involved in the sex-related variability in response to viral infections

APP Processing Induced by Herpes Simplex Virus Type 1 (HSV-1) Yields Several APP Fragments in Human and Rat Neuronal Cells

Lifelong latent infections of the trigeminal ganglion by the neurotropic herpes simplex virus type 1 (HSV-1) are characterized by periodic reactivation. During these episodes, newly produced virions may also reach the central nervous system (CNS), causing productive but generally asymptomatic infections. Epidemiological and experimental findings suggest that HSV-1 might contribute to the pathogenesis of Alzheimer's disease (AD). This multifactorial neurodegenerative disorder is related to an overproduction of amyloid beta (Aβ) and other neurotoxic peptides, which occurs during amyloidogenic endoproteolytic processing of the transmembrane amyloid precursor protein (APP). The aim of our study was to identify the effects of productive HSV-1 infection on APP processing in neuronal cells. We found that infection of SH-SY5Y human neuroblastoma cells and rat cortical neurons is followed by multiple cleavages of APP, which result in the intra- and/or extra-cellular accumulation of various neurotoxic species. These include: i) APP fragments (APP-Fs) of 35 and 45 kDa (APP-F35 and APP-F45) that comprise portions of Aβ; ii) N-terminal APP-Fs that are secreted; iii) intracellular C-terminal APP-Fs; and iv) Aβ1-40 and Aβ1-42. Western blot analysis of infected-cell lysates treated with formic acid suggests that APP-F35 may be an Aβ oligomer. The multiple cleavages of APP that occur in infected cells are produced in part by known components of the amyloidogenic APP processing pathway, i.e., host-cell β-secretase, γ-secretase, and caspase-3-like enzymes. These findings demonstrate that HSV-1 infection of neuronal cells can generate multiple APP fragments with well-documented neurotoxic potentials. It is tempting to speculate that intra- and extracellular accumulation of these species in the CNS resulting from repeated HSV-1 reactivation could, in the presence of other risk factors, play a co-factorial role in the development of AD

A Robust Initialization of Residual Blocks for Effective ResNet Training Without Batch Normalization

Batch Normalization is an essential component of all state-of-the-art neural networks architectures. However, since it introduces many practical issues, much recent research has been devoted to designing normalization-free architectures. In this paper, we show that weights initialization is key to train ResNet-like normalization-free networks. In particular, we propose a slight modification to the summation operation of a block output to the skip-connection branch, so that the whole network is correctly initialized. We show that this modified architecture achieves competitive results on CIFAR-10, CIFAR-100 and ImageNet without further regularization nor algorithmic modifications.Comment: 16 pages (4 pages of supplementary material), 9 figures, 2 tabl