LAGC: Lazily Aggregated Gradient Coding for Straggler-Tolerant and Communication-Efficient Distributed Learning

Gradient-based distributed learning in Parameter Server (PS) computing architectures is subject to random delays due to straggling worker nodes, as well as to possible communication bottlenecks between PS and workers. Solutions have been recently proposed to separately address these impairments based on the ideas of gradient coding, worker grouping, and adaptive worker selection. This paper provides a unified analysis of these techniques in terms of wall-clock time, communication, and computation complexity measures. Furthermore, in order to combine the benefits of gradient coding and grouping in terms of robustness to stragglers with the communication and computation load gains of adaptive selection, novel strategies, named Lazily Aggregated Gradient Coding (LAGC) and Grouped-LAG (G-LAG), are introduced. Analysis and results show that G-LAG provides the best wall-clock time and communication performance, while maintaining a low computational cost, for two representative distributions of the computing times of the worker nodes.Comment: Submitte

Adaptive Randomized Distributed Space-Time Coding in Cooperative MIMO Relay Systems

An adaptive randomized distributed space-time coding (DSTC) scheme and algorithms are proposed for two-hop cooperative MIMO networks. Linear minimum mean square error (MMSE) receivers and an amplify-and-forward (AF) cooperation strategy are considered. In the proposed DSTC scheme, a randomized matrix obtained by a feedback channel is employed to transform the space-time coded matrix at the relay node. Linear MMSE expressions are devised to compute the parameters of the adaptive randomized matrix and the linear receive filter. A stochastic gradient algorithm is also developed to compute the parameters of the adaptive randomized matrix with reduced computational complexity. We also derive the upper bound of the error probability of a cooperative MIMO system employing the randomized space-time coding scheme first. The simulation results show that the proposed algorithms obtain significant performance gains as compared to existing DSTC schemes.Comment: 4 figure

Distributed Space-Time Coding Based on Adjustable Code Matrices for Cooperative MIMO Relaying Systems

An adaptive distributed space-time coding (DSTC) scheme is proposed for two-hop cooperative MIMO networks. Linear minimum mean square error (MMSE) receive filters and adjustable code matrices are considered subject to a power constraint with an amplify-and-forward (AF) cooperation strategy. In the proposed adaptive DSTC scheme, an adjustable code matrix obtained by a feedback channel is employed to transform the space-time coded matrix at the relay node. The effects of the limited feedback and the feedback errors are assessed. Linear MMSE expressions are devised to compute the parameters of the adjustable code matrix and the linear receive filters. Stochastic gradient (SG) and least-squares (LS) algorithms are also developed with reduced computational complexity. An upper bound on the pairwise error probability analysis is derived and indicates the advantage of employing the adjustable code matrices at the relay nodes. An alternative optimization algorithm for the adaptive DSTC scheme is also derived in order to eliminate the need for the feedback. The algorithm provides a fully distributed scheme for the adaptive DSTC at the relay node based on the minimization of the error probability. Simulation results show that the proposed algorithms obtain significant performance gains as compared to existing DSTC schemes.Comment: 6 figure

Registration of Brain MRI/PET Images Based on Adaptive Combination of Intensity and Gradient Field Mutual Information

Traditional mutual information (MI) function aligns two multimodality images with intensity information, lacking spatial information, so that it usually presents many local maxima that can lead to inaccurate registration. Our paper proposes an algorithm of adaptive combination of intensity and gradient field mutual information (ACMI). Gradient code maps (GCM) are constructed by coding gradient field information of corresponding original images. The gradient field MI, calculated from GCMs, can provide complementary properties to intensity MI. ACMI combines intensity MI and gradient field MI with a nonlinear weight function, which can automatically adjust the proportion between two types MI in combination to improve registration. Experimental results demonstrate that ACMI outperforms the traditional MI and it is much less sensitive to reduced resolution or overlap of images

Quality Adaptive Least Squares Trained Filters for Video Compression Artifacts Removal Using a No-reference Block Visibility Metric

Compression artifacts removal is a challenging problem because videos can be compressed at different qualities. In this paper, a least squares approach that is self-adaptive to the visual quality of the input sequence is proposed. For compression artifacts, the visual quality of an image is measured by a no-reference block visibility metric. According to the blockiness visibility of an input image, an appropriate set of filter coefficients that are trained beforehand is selected for optimally removing coding artifacts and reconstructing object details. The performance of the proposed algorithm is evaluated on a variety of sequences compressed at different qualities in comparison to several other deblocking techniques. The proposed method outperforms the others significantly both objectively and subjectively