The Convergence of Sparsified Gradient Methods

Distributed training of massive machine learning models, in particular deep neural networks, via Stochastic Gradient Descent (SGD) is becoming commonplace. Several families of communication-reduction methods, such as quantization, large-batch methods, and gradient sparsification, have been proposed. To date, gradient sparsification methods - where each node sorts gradients by magnitude, and only communicates a subset of the components, accumulating the rest locally - are known to yield some of the largest practical gains. Such methods can reduce the amount of communication per step by up to three orders of magnitude, while preserving model accuracy. Yet, this family of methods currently has no theoretical justification. This is the question we address in this paper. We prove that, under analytic assumptions, sparsifying gradients by magnitude with local error correction provides convergence guarantees, for both convex and non-convex smooth objectives, for data-parallel SGD. The main insight is that sparsification methods implicitly maintain bounds on the maximum impact of stale updates, thanks to selection by magnitude. Our analysis and empirical validation also reveal that these methods do require analytical conditions to converge well, justifying existing heuristics.Comment: NIPS 2018 - Advances in Neural Information Processing Systems; Authors in alphabetic orde

Teriparatide improves early callus formation in distal radial fractures: Analysis of a subgroup of patients within a randomized trial

Background Teriparatide (parathyreoid hormone; PTH 1-34) increases skeletal mass in humans and improves fracture healing in animals. A recent randomized multicenter trial of nonoperated distal radial fractures showed a moderate shortening of the time to restoration of cortical continuity by treatment with 20 μg (low-dose) teriparatide per day, but not with 40 μg (high-dose). As radiographic cortical continuity appears late in the healing process, perhaps too late for clinical relevance, we studied the qualitative appearance of the callus 5 weeks after fracture

Kinetic Energy Release Distributions for C⁺₂ Emission from Multiply Charged C₆₀ and C₇₀ Fullerenes

We present asystematic study of experimental kinetic energy release distributions for the asymmetric fission processes Cq+60 C(iq-1\u3c)+70+ C+2 and C q+70 C(q-1)+60+ C+ 2 for mother ions incharge states q 4-8 produced incollisions with slow highly charged ions. Somewhat to our surprise, we find that the KERD for asymmetric fission from Cq+60 are considerably wider and have larger most likely values than the Cq+70 distributions inthe corresponding charge states when q \u3e 4

Identification and Role of Regulatory Non-Coding RNAs in Listeria monocytogenes

Bacterial regulatory non-coding RNAs control numerous mRNA targets that direct a plethora of biological processes, such as the adaption to environmental changes, growth and virulence. Recently developed high-throughput techniques, such as genomic tiling arrays and RNA-Seq have allowed investigating prokaryotic cis- and trans-acting regulatory RNAs, including sRNAs, asRNAs, untranslated regions (UTR) and riboswitches. As a result, we obtained a more comprehensive view on the complexity and plasticity of the prokaryotic genome biology. Listeria monocytogenes was utilized as a model system for intracellular pathogenic bacteria in several studies, which revealed the presence of about 180 regulatory RNAs in the listerial genome. A regulatory role of non-coding RNAs in survival, virulence and adaptation mechanisms of L. monocytogenes was confirmed in subsequent experiments, thus, providing insight into a multifaceted modulatory function of RNA/mRNA interference. In this review, we discuss the identification of regulatory RNAs by high-throughput techniques and in their functional role in L. monocytogenes

Tumour stromal cells derived from paediatric malignancies display MSC-like properties and impair NK cell cytotoxicity

<p>Abstract</p> <p>Background</p> <p>Tumour growth and metastatic infiltration are favoured by several components of the tumour microenvironment. Bone marrow-derived multipotent mesenchymal stromal cells (MSC) are known to contribute to the tumour stroma. When isolated from healthy bone marrow, MSC exert potent antiproliferative effects on immune effector cells. Due to phenotypic and morphological similarities of MSC and tumour stromal cells (TStrC), we speculated that immunotherapeutic approaches may be hampered if TStrC may still exhibit immunomodulatory properties of MSC.</p> <p>Methods</p> <p>In order to compare immunomodulatory properties of MSC and tumour stromal cells (TStrC), we established and analyzed TStrC cultures from eleven paediatric tumours and MSC preparations from bone marrow aspirates. Immunophenotyping, proliferation assays and NK cell cytotoxicity assays were employed to address the issue.</p> <p>Results</p> <p>While TStrC differed from MSC in terms of plasticity, they shared surface expression of CD105, CD73 and other markers used for MSC characterization. Furthermore, TStrC displayed a strong antiproliferative effect on peripheral blood mononuclear cells (PBMC) in coculture experiments similar to MSC. NK cell cytotoxicity was significantly impaired after co-culture with TStrC and expression of the activating NK cell receptors NKp44 and NKp46 was reduced.</p> <p>Conclusions</p> <p>Our data show that TStrC and MSC share important phenotypic and functional characteristics. The inhibitory effect of TStrC on PBMC and especially on NK cells may facilitate the immune evasion of paediatric tumours.</p

Search for dark matter produced in association with bottom or top quarks in √s = 13 TeV pp collisions with the ATLAS detector

A search for weakly interacting massive particle dark matter produced in association with bottom or top quarks is presented. Final states containing third-generation quarks and miss- ing transverse momentum are considered. The analysis uses 36.1 fb−1 of proton–proton collision data recorded by the ATLAS experiment at √s = 13 TeV in 2015 and 2016. No significant excess of events above the estimated backgrounds is observed. The results are in- terpreted in the framework of simplified models of spin-0 dark-matter mediators. For colour- neutral spin-0 mediators produced in association with top quarks and decaying into a pair of dark-matter particles, mediator masses below 50 GeV are excluded assuming a dark-matter candidate mass of 1 GeV and unitary couplings. For scalar and pseudoscalar mediators produced in association with bottom quarks, the search sets limits on the production cross- section of 300 times the predicted rate for mediators with masses between 10 and 50 GeV and assuming a dark-matter mass of 1 GeV and unitary coupling. Constraints on colour- charged scalar simplified models are also presented. Assuming a dark-matter particle mass of 35 GeV, mediator particles with mass below 1.1 TeV are excluded for couplings yielding a dark-matter relic density consistent with measurements

Genome-wide association identifies nine common variants associated with fasting proinsulin levels and provides new insights into the pathophysiology of type 2 diabetes.

OBJECTIVE: Proinsulin is a precursor of mature insulin and C-peptide. Higher circulating proinsulin levels are associated with impaired β-cell function, raised glucose levels, insulin resistance, and type 2 diabetes (T2D). Studies of the insulin processing pathway could provide new insights about T2D pathophysiology. RESEARCH DESIGN AND METHODS: We have conducted a meta-analysis of genome-wide association tests of ∼2.5 million genotyped or imputed single nucleotide polymorphisms (SNPs) and fasting proinsulin levels in 10,701 nondiabetic adults of European ancestry, with follow-up of 23 loci in up to 16,378 individuals, using additive genetic models adjusted for age, sex, fasting insulin, and study-specific covariates. RESULTS: Nine SNPs at eight loci were associated with proinsulin levels (P < 5 × 10(-8)). Two loci (LARP6 and SGSM2) have not been previously related to metabolic traits, one (MADD) has been associated with fasting glucose, one (PCSK1) has been implicated in obesity, and four (TCF7L2, SLC30A8, VPS13C/C2CD4A/B, and ARAP1, formerly CENTD2) increase T2D risk. The proinsulin-raising allele of ARAP1 was associated with a lower fasting glucose (P = 1.7 × 10(-4)), improved β-cell function (P = 1.1 × 10(-5)), and lower risk of T2D (odds ratio 0.88; P = 7.8 × 10(-6)). Notably, PCSK1 encodes the protein prohormone convertase 1/3, the first enzyme in the insulin processing pathway. A genotype score composed of the nine proinsulin-raising alleles was not associated with coronary disease in two large case-control datasets. CONCLUSIONS: We have identified nine genetic variants associated with fasting proinsulin. Our findings illuminate the biology underlying glucose homeostasis and T2D development in humans and argue against a direct role of proinsulin in coronary artery disease pathogenesis