Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication

Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes are becoming increasingly relevant. A major issue in distributed training is the limited communication bandwidth between contributing nodes or prohibitive communication cost in general. These challenges become even more pressing, as the number of computation nodes increases. To counteract this development we propose sparse binary compression (SBC), a compression framework that allows for a drastic reduction of communication cost for distributed training. SBC combines existing techniques of communication delay and gradient sparsification with a novel binarization method and optimal weight update encoding to push compression gains to new limits. By doing so, our method also allows us to smoothly trade-off gradient sparsity and temporal sparsity to adapt to the requirements of the learning task. Our experiments show, that SBC can reduce the upstream communication on a variety of convolutional and recurrent neural network architectures by more than four orders of magnitude without significantly harming the convergence speed in terms of forward-backward passes. For instance, we can train ResNet50 on ImageNet in the same number of iterations to the baseline accuracy, using $\times 3531$ less bits or train it to a $1\%$ lower accuracy using $\times 37208$ less bits. In the latter case, the total upstream communication required is cut from 125 terabytes to 3.35 gigabytes for every participating client

Spontaneous polyploidisation of interspecific and intersectional Pelargonium hybrids during embryo rescue

Modern Pelargonium crispum hybrids (section Pelargonium) show low genetic and phenotypic variation due to the domestication effect. Species of the sections Cortusina, Ligularia, and Pelargonium are potential breeding partners at the diploid level (2n = 2x = 22). Five P. × crispum cultivars were used as seed parents and pollinated with one genotype of P. grandiflorum (section Pelargonium) and three genotypes of P. fulgidum (section Ligularia). In both combinations, embryo rescue was necessary. Embryos were rescued and cultured on Murashige & Skoog medium supplemented with phytohormones. After callus and adventitious shoot regeneration 15 viable interspecific hybrids were obtained from crossbreeding with P. grandiflorum and 11 intersectional hybrids from crossings with P. fulgidum, respectively. The hybrids were cultivated in the greenhouse until flowering. Their hybrid character was evident due to the intermediate morphological traits. Molecular investigations using dp-RAPD analysis confirmed this. Within the F1 population P. × crispum with P. grandiflorum three hybrids and after crossing with P. fulgidum one hybrid possessed larger flowers and fully developed anthers, respectively. Their ploidy level was confirmed as tetraploid using flow cytometry. Therefore, a spontaneous polyploidisation occurred during in vitro regeneration. The tetraploid F1 hybrids are fertile and could be used for further breeding

Strigolactone biosynthesis is evolutionarily conserved, regulated by phosphate starvation and contributes to resistance against phytopathogenic fungi in a moss, Physcomitrella patens

In seed plants, strigolactones (SLs) regulate architecture and induce mycorrhizal symbiosis in response to environmental cues. SLs are formed by combined activity of the carotenoid cleavage dioxygenases (CCDs) 7 and 8 from 9-cis-β-carotene, leading to carlactone that is converted by cytochromes P450 (clade 711; MAX1 in Arabidopsis) into various SLs. As Physcomitrella patens possesses CCD7 and CCD8 homologs but lacks MAX1, we investigated if PpCCD7 together with PpCCD8 form carlactone and how deletion of these enzymes influences growth and interactions with the environment. We investigated the enzymatic activity of PpCCD7 and PpCCD8 in vitro, identified the formed products by high performance liquid chromatography (HPLC) and LC-MS, and generated and analysed ΔCCD7 and ΔCCD8 mutants. We defined enzymatic activity of PpCCD7 as a stereospecific 9-cis-CCD and PpCCD8 as a carlactone synthase. ΔCCD7 and ΔCCD8 lines showed enhanced caulonema growth, which was revertible by adding the SL analogue GR24 or carlactone. Wild-type (WT) exudates induced seed germination in Orobanche ramosa. This activity was increased upon phosphate starvation and abolished in exudates of both mutants. Furthermore, both mutants showed increased susceptibility to phytopathogenic fungi. Our study reveals the deep evolutionary conservation of SL biosynthesis, SL function, and its regulation by biotic and abiotic cues.Deutsche Forschungsgemeinschaft AL892/1-4Academy of Finland 125312

Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning

The success of convolutional neural networks (CNNs) in various applications is accompanied by a significant increase in computation and parameter storage costs. Recent efforts to reduce these overheads involve pruning and compressing the weights of various layers while at the same time aiming to not sacrifice performance. In this paper, we propose a novel criterion for CNN pruning inspired by neural network interpretability: The most relevant units, i.e. weights or filters, are automatically found using their relevance scores obtained from concepts of explainable AI (XAI). By exploring this idea, we connect the lines of interpretability and model compression research. We show that our proposed method can efficiently prune CNN models in transfer-learning setups in which networks pre-trained on large corpora are adapted to specialized tasks. The method is evaluated on a broad range of computer vision datasets. Notably, our novel criterion is not only competitive or better compared to state-of-the-art pruning criteria when successive retraining is performed, but clearly outperforms these previous criteria in the resource-constrained application scenario in which the data of the task to be transferred to is very scarce and one chooses to refrain from fine-tuning. Our method is able to compress the model iteratively while maintaining or even improving accuracy. At the same time, it has a computational cost in the order of gradient computation and is comparatively simple to apply without the need for tuning hyperparameters for pruning.Comment: 25 pages + 5 supplementary pages, 13 figures, 6 table

No association between major depression with and without childhood adversity and the stress hormone copeptin

Background: Adverse childhood experiences (ACE) are associated with an increased risk of major depressive disorder (MDD) and hypothalamic-pituitary-adrenal (HPA) axis dysregulation. Within the HPA axis, corticotropin-releasing hormone and vasopressin (AVP) synergistically stimulate the release of adrenocorticotropic hormone, which promotes cortisol release. The cleavage product copeptin is produced during AVP synthesis and is a surrogate marker of AVP release. Children with ACE and young adults with depressive symptoms have higher levels of copeptin than healthy controls. Objective: To uncover the effects of MDD and ACE on copeptin levels in adult females. Methods: We recruited 94 women (mean age: 34.0 +/- 3.6 years): 23 with MDD and ACE, 24 with MDD without ACE, 22 with ACE without MDD, and 25 healthy controls. ACE was defined as repeated sexual or physical abuse at least once a month over at least one year before the age of 18 years. MDD was defined by the DSM-IV criteria. Copeptin plasma levels were measured with an immunoluminometric assay. Results: The four groups did not differ in demographic variables. We found a significant negative correlation between body mass index (BMI) and copeptin plasma levels (r = -.21; p = .045). Copeptin plasma levels did not differ between the four groups after controlling for BMI. Conclusion: Neither MDD nor ACE was associated with altered plasma copeptin levels. Thus, copeptin does not seem to play a major role in MDD and ACE in adult females

Peptide Fingerprinting of Alzheimer's Disease in Cerebrospinal Fluid: Identification and Prospective Evaluation of New Synaptic Biomarkers

<p><b>Background:</b> Today, dementias are diagnosed late in the course of disease. Future treatments have to start earlier in the disease process to avoid disability requiring new diagnostic tools. The objective of this study is to develop a new method for the differential diagnosis and identification of new biomarkers of Alzheimer's disease (AD) using capillary-electrophoresis coupled to mass-spectrometry (CE-MS) and to assess the potential of early diagnosis of AD.</p> <p><b>Methods and Findings:</b> Cerebrospinal fluid (CSF) of 159 out-patients of a memory-clinic at a University Hospital suffering from neurodegenerative disorders and 17 cognitively-healthy controls was used to create differential peptide pattern for dementias and prospective blinded-comparison of sensitivity and specificity for AD diagnosis against the Criterion standard in a naturalistic prospective sample of patients. Sensitivity and specificity of the new method compared to standard diagnostic procedures and identification of new putative biomarkers for AD was the main outcome measure. CE-MS was used to reliably detect 1104 low-molecular-weight peptides in CSF. Training-sets of patients with clinically secured sporadic Alzheimer's disease, frontotemporal dementia, and cognitively healthy controls allowed establishing discriminative biomarker pattern for diagnosis of AD. This pattern was already detectable in patients with mild cognitive impairment (MCI). The AD-pattern was tested in a prospective sample of patients (n = 100) and AD was diagnosed with a sensitivity of 87% and a specificity of 83%. Using CSF measurements of beta-amyloid1-42, total-tau, and phospho(181)-tau, AD-diagnosis had a sensitivity of 88% and a specificity of 67% in the same sample. Sequence analysis of the discriminating biomarkers identified fragments of synaptic proteins like proSAAS, apolipoprotein J, neurosecretory protein VGF, phospholemman, and chromogranin A.</p> <p><b>Conclusions:</b> The method may allow early differential diagnosis of various dementias using specific peptide fingerprints and identification of incipient AD in patients suffering from MCI. Identified biomarkers facilitate face validity for the use in AD diagnosis.</p&gt

Early onset of treatment effects with oral risperidone

BACKGROUND: The dogma of a delayed onset of antipsychotic treatment effects has been maintained over the past decades. However, recent studies have challenged this concept. We therefore performed an analysis of the onset of antipsychotic treatment effects in a sample of acutely decompensated patients with schizophrenia. METHODS: In this observational study, 48 inpatients with acutely decompensated schizophrenia were offered antipsychotic treatment with oral risperidone. PANSS-ratings were obtained on day 0, day 1, day 3, day 7 and day 14. RESULTS: Significant effects of treatment were already present on day 1 and continued throughout the study. The PANSS positive subscore and the PANSS total score improved significantly more than the PANSS negative subscore. CONCLUSION: Our results are consistent with the growing number of studies suggesting an early onset of antipsychotic treatment effects. However, non-pharmacological effects of treatment also need to be taken into consideration