Genuinely Distributed Byzantine Machine Learning

Machine Learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component failures, which can be all encompassed within the spectrum of a Byzantine behavior. Several approaches have been proposed recently to tolerate Byzantine workers. Yet all require trusting a central parameter server. We initiate in this paper the study of the ``general'' Byzantine-resilient distributed machine learning problem where no individual component is trusted. We show that this problem can be solved in an asynchronous system, despite the presence of $\frac{1}{3}$ Byzantine parameter servers and $\frac{1}{3}$ Byzantine workers (which is optimal). We present a new algorithm, ByzSGD, which solves the general Byzantine-resilient distributed machine learning problem by relying on three major schemes. The first, Scatter/Gather, is a communication scheme whose goal is to bound the maximum drift among models on correct servers. The second, Distributed Median Contraction (DMC), leverages the geometric properties of the median in high dimensional spaces to bring parameters within the correct servers back close to each other, ensuring learning convergence. The third, Minimum-Diameter Averaging (MDA), is a statistically-robust gradient aggregation rule whose goal is to tolerate Byzantine workers. MDA requires loose bound on the variance of non-Byzantine gradient estimates, compared to existing alternatives (e.g., Krum). Interestingly, ByzSGD ensures Byzantine resilience without adding communication rounds (on a normal path), compared to vanilla non-Byzantine alternatives. ByzSGD requires, however, a larger number of messages which, we show, can be reduced if we assume synchrony.Comment: This is a merge of arXiv:1905.03853 and arXiv:1911.07537; arXiv:1911.07537 will be retracte

Putting It Together: AIDS and the Millennium Development Goals

Failure to halt and reverse the HIV/AIDS epidemic, say the authors, will continue to jeopardize progress on achieving a wide range of the MDGs

LocTree3 prediction of localization

The prediction of protein sub-cellular localization is an important step toward elucidating protein function. For each query protein sequence, LocTree2 applies machine learning (profile kernel SVM) to predict the native sub-cellular localization in 18 classes for eukaryotes, in six for bacteria and in three for archaea. The method outputs a score that reflects the reliability of each prediction. LocTree2 has performed on par with or better than any other state-of-the-art method. Here, we report the availability of LocTree3 as a public web server. The server includes the machine learning-based LocTree2 and improves over it through the addition of homology-based inference. Assessed on sequence-unique data, LocTree3 reached an 18-state accuracy Q18 = 80 ± 3% for eukaryotes and a six-state accuracy Q6 = 89 ± 4% for bacteria. The server accepts submissions ranging from single protein sequences to entire proteomes. Response time of the unloaded server is about 90 s for a 300-residue eukaryotic protein and a few hours for an entire eukaryotic proteome not considering the generation of the alignments. For over 1000 entirely sequenced organisms, the predictions are directly available as downloads. The web server is available at http://www.rostlab.org/services/loctree3

The V471A polymorphism in autophagy-related gene ATG7 modifies age at onset specifically in Italian Huntington disease patients

The cause of Huntington disease (HD) is a polyglutamine repeat expansion of more than 36 units in the huntingtin protein, which is inversely correlated with the age at onset of the disease. However, additional genetic factors are believed to modify the course and the age at onset of HD. Recently, we identified the V471A polymorphism in the autophagy-related gene ATG7, a key component of the autophagy pathway that plays an important role in HD pathogenesis, to be associated with the age at onset in a large group of European Huntington disease patients. To confirm this association in a second independent patient cohort, we analysed the ATG7 V471A polymorphism in additional 1,464 European HD patients of the “REGISTRY” cohort from the European Huntington Disease Network (EHDN). In the entire REGISTRY cohort we could not confirm a modifying effect of the ATG7 V471A polymorphism. However, analysing a modifying effect of ATG7 in these REGISTRY patients and in patients of our previous HD cohort according to their ethnic origin, we identified a significant effect of the ATG7 V471A polymorphism on the HD age at onset only in the Italian population (327 patients). In these Italian patients, the polymorphism is associated with a 6-years earlier disease onset and thus seems to have an aggravating effect. We could specify the role of ATG7 as a genetic modifier for HD particularly in the Italian population. This result affirms the modifying influence of the autophagic pathway on the course of HD, but also suggests population-specific modifying mechanisms in HD pathogenesis

Identification of genetic variants associated with Huntington's disease progression: a genome-wide association study

Background Huntington's disease is caused by a CAG repeat expansion in the huntingtin gene, HTT. Age at onset has been used as a quantitative phenotype in genetic analysis looking for Huntington's disease modifiers, but is hard to define and not always available. Therefore, we aimed to generate a novel measure of disease progression and to identify genetic markers associated with this progression measure. Methods We generated a progression score on the basis of principal component analysis of prospectively acquired longitudinal changes in motor, cognitive, and imaging measures in the 218 indivduals in the TRACK-HD cohort of Huntington's disease gene mutation carriers (data collected 2008–11). We generated a parallel progression score using data from 1773 previously genotyped participants from the European Huntington's Disease Network REGISTRY study of Huntington's disease mutation carriers (data collected 2003–13). We did a genome-wide association analyses in terms of progression for 216 TRACK-HD participants and 1773 REGISTRY participants, then a meta-analysis of these results was undertaken. Findings Longitudinal motor, cognitive, and imaging scores were correlated with each other in TRACK-HD participants, justifying use of a single, cross-domain measure of disease progression in both studies. The TRACK-HD and REGISTRY progression measures were correlated with each other (r=0·674), and with age at onset (TRACK-HD, r=0·315; REGISTRY, r=0·234). The meta-analysis of progression in TRACK-HD and REGISTRY gave a genome-wide significant signal (p=1·12 × 10−10) on chromosome 5 spanning three genes: MSH3, DHFR, and MTRNR2L2. The genes in this locus were associated with progression in TRACK-HD (MSH3 p=2·94 × 10−8 DHFR p=8·37 × 10−7 MTRNR2L2 p=2·15 × 10−9) and to a lesser extent in REGISTRY (MSH3 p=9·36 × 10−4 DHFR p=8·45 × 10−4 MTRNR2L2 p=1·20 × 10−3). The lead single nucleotide polymorphism (SNP) in TRACK-HD (rs557874766) was genome-wide significant in the meta-analysis (p=1·58 × 10−8), and encodes an aminoacid change (Pro67Ala) in MSH3. In TRACK-HD, each copy of the minor allele at this SNP was associated with a 0·4 units per year (95% CI 0·16–0·66) reduction in the rate of change of the Unified Huntington's Disease Rating Scale (UHDRS) Total Motor Score, and a reduction of 0·12 units per year (95% CI 0·06–0·18) in the rate of change of UHDRS Total Functional Capacity score. These associations remained significant after adjusting for age of onset. Interpretation The multidomain progression measure in TRACK-HD was associated with a functional variant that was genome-wide significant in our meta-analysis. The association in only 216 participants implies that the progression measure is a sensitive reflection of disease burden, that the effect size at this locus is large, or both. Knockout of Msh3 reduces somatic expansion in Huntington's disease mouse models, suggesting this mechanism as an area for future therapeutic investigation

Abstract Neural Networks Letter Cogent confabulation

A new model of vertebrate cognition is introduced: maximization of cogency p(abgdj3). This model is shown to be a direct generalization of Aristotelian logic, and to be rigorously related to a calculable quantity. A key aspect of this model is that in Aristotelian logic information environments it functions logically. However, in non-Aristotelian environments, instead of finding the conclusion with the highest probability of being true (a popular past model of cognition); this model instead functions in the manner of the ‘duck test; ’ by finding that conclusion which is most supportive of the truth of the assumed facts

Confabulation Theory A Synopsis

A theory of the cognitive function of human cerebral cortex is sketched. 1

Thinking

A theory of the function of mammalian cerebral cortex is sketched. Keywords cerebral cortex theory cortical information processing feature attractor neural network antecedent support neural network confabulation mammalian law of thought brain command loop artificial intelligence