LAGC: Lazily Aggregated Gradient Coding for Straggler-Tolerant and Communication-Efficient Distributed Learning

Gradient-based distributed learning in Parameter Server (PS) computing architectures is subject to random delays due to straggling worker nodes, as well as to possible communication bottlenecks between PS and workers. Solutions have been recently proposed to separately address these impairments based on the ideas of gradient coding, worker grouping, and adaptive worker selection. This paper provides a unified analysis of these techniques in terms of wall-clock time, communication, and computation complexity measures. Furthermore, in order to combine the benefits of gradient coding and grouping in terms of robustness to stragglers with the communication and computation load gains of adaptive selection, novel strategies, named Lazily Aggregated Gradient Coding (LAGC) and Grouped-LAG (G-LAG), are introduced. Analysis and results show that G-LAG provides the best wall-clock time and communication performance, while maintaining a low computational cost, for two representative distributions of the computing times of the worker nodes.Comment: Submitte

Cloud-Edge Non-Orthogonal Transmission for Fog Networks with Delayed CSI at the Cloud

In a Fog Radio Access Network (F-RAN), the cloud processor (CP) collects channel state information (CSI) from the edge nodes (ENs) over fronthaul links. As a result, the CSI at the cloud is generally affected by an error due to outdating. In this work, the problem of content delivery based on fronthaul transmission and edge caching is studied from an information-theoretic perspective in the high signal-to-noise ratio (SNR) regime. For the set-up under study, under the assumption of perfect CSI, prior work has shown the (approximate or exact) optimality of a scheme in which the ENs transmit information received from the cloud and cached contents over orthogonal resources. In this work, it is demonstrated that a non-orthogonal transmission scheme is able to substantially improve the latency performance in the presence of imperfect CSI at the cloud.Comment: 5 pages, 4 figures, submitte

Fundamental Limits of Cloud and Cache-Aided Interference Management with Multi-Antenna Edge Nodes

In fog-aided cellular systems, content delivery latency can be minimized by jointly optimizing edge caching and transmission strategies. In order to account for the cache capacity limitations at the Edge Nodes (ENs), transmission generally involves both fronthaul transfer from a cloud processor with access to the content library to the ENs, as well as wireless delivery from the ENs to the users. In this paper, the resulting problem is studied from an information-theoretic viewpoint by making the following practically relevant assumptions: 1) the ENs have multiple antennas; 2) only uncoded fractional caching is allowed; 3) the fronthaul links are used to send fractions of contents; and 4) the ENs are constrained to use one-shot linear precoding on the wireless channel. Assuming offline proactive caching and focusing on a high signal-to-noise ratio (SNR) latency metric, the optimal information-theoretic performance is investigated under both serial and pipelined fronthaul-edge transmission modes. The analysis characterizes the minimum high-SNR latency in terms of Normalized Delivery Time (NDT) for worst-case users' demands. The characterization is exact for a subset of system parameters, and is generally optimal within a multiplicative factor of 3/2 for the serial case and of 2 for the pipelined case. The results bring insights into the optimal interplay between edge and cloud processing in fog-aided wireless networks as a function of system resources, including the number of antennas at the ENs, the ENs' cache capacity and the fronthaul capacity.Comment: 34 pages, 15 figures, submitte