194 research outputs found
Compressed Distributed Gradient Descent: Communication-Efficient Consensus over Networks
Network consensus optimization has received increasing attention in recent
years and has found important applications in many scientific and engineering
fields. To solve network consensus optimization problems, one of the most
well-known approaches is the distributed gradient descent method (DGD).
However, in networks with slow communication rates, DGD's performance is
unsatisfactory for solving high-dimensional network consensus problems due to
the communication bottleneck. This motivates us to design a
communication-efficient DGD-type algorithm based on compressed information
exchanges. Our contributions in this paper are three-fold: i) We develop a
communication-efficient algorithm called amplified-differential compression DGD
(ADC-DGD) and show that it converges under {\em any} unbiased compression
operator; ii) We rigorously prove the convergence performances of ADC-DGD and
show that they match with those of DGD without compression; iii) We reveal an
interesting phase transition phenomenon in the convergence speed of ADC-DGD.
Collectively, our findings advance the state-of-the-art of network consensus
optimization theory.Comment: 11 pages, 11 figures, IEEE INFOCOM 201
Spatial CUSUM for Signal Region Detection
Detecting weak clustered signal in spatial data is important but challenging
in applications such as medical image and epidemiology. A more efficient
detection algorithm can provide more precise early warning, and effectively
reduce the decision risk and cost. To date, many methods have been developed to
detect signals with spatial structures. However, most of the existing methods
are either too conservative for weak signals or computationally too intensive.
In this paper, we consider a novel method named Spatial CUSUM (SCUSUM), which
employs the idea of the CUSUM procedure and false discovery rate controlling.
We develop theoretical properties of the method which indicates that
asymptotically SCUSUM can reach high classification accuracy. In the simulation
study, we demonstrate that SCUSUM is sensitive to weak spatial signals. This
new method is applied to a real fMRI dataset as illustration, and more
irregular weak spatial signals are detected in the images compared to some
existing methods, including the conventional FDR, FDR and scan statistics
Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors
Network-distributed optimization has attracted significant attention in recent years due to its ever-increasing applications. However, the classic decentralized gradient descent (DGD) algorithm is communication-inefficient for large-scale and high-dimensional network-distributed optimization problems. To address this challenge, many compressed DGD-based algorithms have been proposed. However, most of the existing works have high complexity and assume compressors with bounded noise power. To overcome these limitations, in this paper, we propose a new differential-coded compressed DGD (DC-DGD) algorithm. The key features of DC-DGD include: i) DC-DGD works with general SNR-constrained compressors, relaxing the bounded noise power assumption; ii) The differential-coded design entails the same convergence rate as the original DGD algorithm; and iii) DC-DGD has the same low-complexity structure as the original DGD due to a {\em self-noise-reduction effect}. Moreover, the above features inspire us to develop a hybrid compression scheme that offers a systematic mechanism to minimize the communication cost. Finally, we conduct extensive experiments to verify the efficacy of the proposed DC-DGD and hybrid compressor
Optimal Stratification and Allocation for the June Agricultural Survey
A computational approach to optimal multivariate designs with respect to stratification and allocation is investigated under the assumptions of fixed total allocation, known number of strata, and the availability of administrative data correlated with thevariables of interest under coefficient-of-variation constraints. This approach uses a penalized objective function that is optimized by simulated annealing through exchanging sampling units and sample allocations among strata. Computational speed is improved through the use of a computationally efficient machine learning method such as K-means to create an initial stratification close to the optimal stratification. The numeric stability of the algorithm has been investigated and parallel processing has been employed where appropriate. Results are presented for both simulated data and USDA’s June Agricultural Survey. An R package has also been made available for evaluation
- …