D2^2: Decentralized Training over Decentralized Data

While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}. Ironically, recent analysis of decentralized parallel stochastic gradient descent (D-PSGD) relies on the assumption that the data hosted on different workers are {\em not too different}. In this paper, we ask the question: {\em Can we design a decentralized parallel stochastic gradient descent algorithm that is less sensitive to the data variance across workers?} In this paper, we present D$^2$, a novel decentralized parallel stochastic gradient descent algorithm designed for large data variance \xr{among workers} (imprecisely, "decentralized" data). The core of D$^2$ is a variance blackuction extension of the standard D-PSGD algorithm, which improves the convergence rate from $O\left({\sigma \over \sqrt{nT}} + {(n\zeta^2)^{\frac{1}{3}} \over T^{2/3}}\right)$ to $O\left({\sigma \over \sqrt{nT}}\right)$ where $\zeta^{2}$ denotes the variance among data on different workers. As a result, D$^2$ is robust to data variance among workers. We empirically evaluated D$^2$ on image classification tasks where each worker has access to only the data of a limited set of labels, and find that D$^2$ significantly outperforms D-PSGD

On the multiplicity of Laplacian eigenvalues of graphs

summary:In this paper we investigate the effect on the multiplicity of Laplacian eigenvalues of two disjoint connected graphs when adding an edge between them. As an application of the result, the multiplicity of 1 as a Laplacian eigenvalue of trees is also considered

Vulnerability Analysis of Soft Caving Tunnel Support System and Surrounding Rock Optimal Control Technology Research

The vulnerability assessment model, composed by 11 vulnerability factors, is established with the introduction of the concept of “vulnerability” into the assessment of tunnel support system. Analytic hierarchy process is utilized to divide these 11 factors into human attributes and natural attributes, and define the weight of these factors for the model. The “vulnerability” applied io the assessment of the tunnel support system model is reached. The vulnerability assessment model was used for evaluating and modifying the haulage tunnel #3207 of Bo-fang mine panel #2. The results decreased the vulnerability of the tunnel support system and demonstrated acceptable effects. Furthermore, the results show that the impact of human attributes on tunnel support systems is dramatic under the condition that natural attributes are permanent, and the “vulnerability” is exactly a notable factor to manifest the transformation during this process. The results also indicate that optimizing human attributes can attenuate vulnerability in tunnel support systems. As a result, enhancement of stability of tunnel support systems can be achieved