Byzantine Stochastic Gradient Descent

This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of the $m$ machines which allegedly compute stochastic gradients every iteration, an $\alpha$-fraction are Byzantine, and can behave arbitrarily and adversarially. Our main result is a variant of stochastic gradient descent (SGD) which finds $\varepsilon$-approximate minimizers of convex functions in $T = \tilde{O}\big( \frac{1}{\varepsilon^2 m} + \frac{\alpha^2}{\varepsilon^2} \big)$ iterations. In contrast, traditional mini-batch SGD needs $T = O\big( \frac{1}{\varepsilon^2 m} \big)$ iterations, but cannot tolerate Byzantine failures. Further, we provide a lower bound showing that, up to logarithmic factors, our algorithm is information-theoretically optimal both in terms of sampling complexity and time complexity

Functional Brain Imaging with Multi-Objective Multi-Modal Evolutionary Optimization

Functional brain imaging is a source of spatio-temporal data mining problems. A new framework hybridizing multi-objective and multi-modal optimization is proposed to formalize these data mining problems, and addressed through Evolutionary Computation (EC). The merits of EC for spatio-temporal data mining are demonstrated as the approach facilitates the modelling of the experts' requirements, and flexibly accommodates their changing goals

Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning

Conventional ensemble learning combines students in the space domain. In this paper, however, we combine students in the time domain and call it time-domain ensemble learning. We analyze, compare, and discuss the generalization performances regarding time-domain ensemble learning of both a linear model and a nonlinear model. Analyzing in the framework of online learning using a statistical mechanical method, we show the qualitatively different behaviors between the two models. In a linear model, the dynamical behaviors of the generalization error are monotonic. We analytically show that time-domain ensemble learning is twice as effective as conventional ensemble learning. Furthermore, the generalization error of a nonlinear model features nonmonotonic dynamical behaviors when the learning rate is small. We numerically show that the generalization performance can be improved remarkably by using this phenomenon and the divergence of students in the time domain.Comment: 11 pages, 7 figure

Gene Function Classification Using Bayesian Models with Hierarchy-Based Priors

We investigate the application of hierarchical classification schemes to the annotation of gene function based on several characteristics of protein sequences including phylogenic descriptors, sequence based attributes, and predicted secondary structure. We discuss three Bayesian models and compare their performance in terms of predictive accuracy. These models are the ordinary multinomial logit (MNL) model, a hierarchical model based on a set of nested MNL models, and a MNL model with a prior that introduces correlations between the parameters for classes that are nearby in the hierarchy. We also provide a new scheme for combining different sources of information. We use these models to predict the functional class of Open Reading Frames (ORFs) from the E. coli genome. The results from all three models show substantial improvement over previous methods, which were based on the C5 algorithm. The MNL model using a prior based on the hierarchy outperforms both the non-hierarchical MNL model and the nested MNL model. In contrast to previous attempts at combining these sources of information, our approach results in a higher accuracy rate when compared to models that use each data source alone. Together, these results show that gene function can be predicted with higher accuracy than previously achieved, using Bayesian models that incorporate suitable prior information

GluRδ2 Expression in the Mature Cerebellum of Hotfoot Mice Promotes Parallel Fiber Synaptogenesis and Axonal Competition

Glutamate receptor delta 2 (GluRdelta2) is selectively expressed in the cerebellum, exclusively in the spines of the Purkinje cells (PCs) that are in contact with parallel fibers (PFs). Although its structure is similar to ionotropic glutamate receptors, it has no channel function and its ligand is unknown. The GluRdelta2-null mice, such as knockout and hotfoot have profoundly altered cerebellar circuitry, which causes ataxia and impaired motor learning. Notably, GluRdelta2 in PC-PF synapses regulates their maturation and strengthening and induces long term depression (LTD). In addition, GluRdelta2 participates in the highly territorial competition between the two excitatory inputs to the PC; the climbing fiber (CF), which innervates the proximal dendritic compartment, and the PF, which is connected to spiny distal branchlets. Recently, studies have suggested that GluRdelta2 acts as an adhesion molecule in PF synaptogenesis. Here, we provide in vivo and in vitro evidence that supports this hypothesis. Through lentiviral rescue in hotfoot mice, we noted a recovery of PC-PF contacts in the distal dendritic domain. In the proximal domain, we observed the formation of new spines that were innervated by PFs and a reduction in contact with the CF; ie, the pattern of innervation in the PC shifted to favor the PF input. Moreover, ectopic expression of GluRdelta2 in HEK293 cells that were cocultured with granule cells or in cerebellar Golgi cells in the mature brain induced the formation of new PF contacts. Collectively, our observations show that GluRdelta2 is an adhesion molecule that induces the formation of PF contacts independently of its cellular localization and promotes heterosynaptic competition in the PC proximal dendritic domain

Estimation of bubble dynamics in the Chinese real estate market: a state space model

This paper analyses the existence of a bubble in the Chinese real estate market and examines its driving factors with a state-space model. The model considers macroeconomic and real estate time series variables as inputs and employs a Kalman filter to obtain an estimated fundamental price using demand and supply for Chinese real estate. We then measure the deviation between actual and estimated fundamental real estate prices to test for the existence of a bubble. We find evidence for the existence of a bubble especially post 2010, when the deviation ratio is found to be significantly higher with a peak of 80% in 2012. Our estimation of overvaluation is generally much higher than in other studies