Scaling-up Distributed Processing of Data Streams for Machine Learning

Emerging applications of machine learning in numerous areas involve continuous gathering of and learning from streams of data. Real-time incorporation of streaming data into the learned models is essential for improved inference in these applications. Further, these applications often involve data that are either inherently gathered at geographically distributed entities or that are intentionally distributed across multiple machines for memory, computational, and/or privacy reasons. Training of models in this distributed, streaming setting requires solving stochastic optimization problems in a collaborative manner over communication links between the physical entities. When the streaming data rate is high compared to the processing capabilities of compute nodes and/or the rate of the communications links, this poses a challenging question: how can one best leverage the incoming data for distributed training under constraints on computing capabilities and/or communications rate? A large body of research has emerged in recent decades to tackle this and related problems. This paper reviews recently developed methods that focus on large-scale distributed stochastic optimization in the compute- and bandwidth-limited regime, with an emphasis on convergence analysis that explicitly accounts for the mismatch between computation, communication and streaming rates. In particular, it focuses on methods that solve: (i) distributed stochastic convex problems, and (ii) distributed principal component analysis, which is a nonconvex problem with geometric structure that permits global convergence. For such methods, the paper discusses recent advances in terms of distributed algorithmic designs when faced with high-rate streaming data. Further, it reviews guarantees underlying these methods, which show there exist regimes in which systems can learn from distributed, streaming data at order-optimal rates.Comment: 45 pages, 9 figures; preprint of a journal paper published in Proceedings of the IEEE (Special Issue on Optimization for Data-driven Learning and Control

Video Streaming in Distributed Erasure-coded Storage Systems: Stall Duration Analysis

The demand for global video has been burgeoning across industries. With the expansion and improvement of video-streaming services, cloud-based video is evolving into a necessary feature of any successful business for reaching internal and external audiences. This paper considers video streaming over distributed systems where the video segments are encoded using an erasure code for better reliability thus being the first work to our best knowledge that considers video streaming over erasure-coded distributed cloud systems. The download time of each coded chunk of each video segment is characterized and ordered statistics over the choice of the erasure-coded chunks is used to obtain the playback time of different video segments. Using the playback times, bounds on the moment generating function on the stall duration is used to bound the mean stall duration. Moment generating function based bounds on the ordered statistics are also used to bound the stall duration tail probability which determines the probability that the stall time is greater than a pre-defined number. These two metrics, mean stall duration and the stall duration tail probability, are important quality of experience (QoE) measures for the end users. Based on these metrics, we formulate an optimization problem to jointly minimize the convex combination of both the QoE metrics averaged over all requests over the placement and access of the video content. The non-convex problem is solved using an efficient iterative algorithm. Numerical results show significant improvement in QoE metrics for cloud-based video as compared to the considered baselines.Comment: 18 pages, accepted to IEEE/ACM Transactions on Networkin

Near-Optimal Hybrid Processing for Massive MIMO Systems via Matrix Decomposition

For the practical implementation of massive multiple-input multiple-output (MIMO) systems, the hybrid processing (precoding/combining) structure is promising to reduce the high cost rendered by large number of RF chains of the traditional processing structure. The hybrid processing is performed through low-dimensional digital baseband processing combined with analog RF processing enabled by phase shifters. We propose to design hybrid RF and baseband precoders/combiners for multi-stream transmission in point-to-point massive MIMO systems, by directly decomposing the pre-designed unconstrained digital precoder/combiner of a large dimension. The constant amplitude constraint of analog RF processing results in the matrix decomposition problem non-convex. Based on an alternate optimization technique, the non-convex matrix decomposition problem can be decoupled into a series of convex sub-problems and effectively solved by restricting the phase increment of each entry in the RF precoder/combiner within a small vicinity of its preceding iterate. A singular value decomposition based technique is proposed to secure an initial point sufficiently close to the global solution of the original non-convex problem. Through simulation, the convergence of the alternate optimization for such a matrix decomposition based hybrid processing (MD-HP) scheme is examined, and the performance of the MD-HP scheme is demonstrated to be near-optimal

Recent Advances in Cloud Radio Access Networks: System Architectures, Key Techniques, and Open Issues

As a promising paradigm to reduce both capital and operating expenditures, the cloud radio access network (C-RAN) has been shown to provide high spectral efficiency and energy efficiency. Motivated by its significant theoretical performance gains and potential advantages, C-RANs have been advocated by both the industry and research community. This paper comprehensively surveys the recent advances of C-RANs, including system architectures, key techniques, and open issues. The system architectures with different functional splits and the corresponding characteristics are comprehensively summarized and discussed. The state-of-the-art key techniques in C-RANs are classified as: the fronthaul compression, large-scale collaborative processing, and channel estimation in the physical layer; and the radio resource allocation and optimization in the upper layer. Additionally, given the extensiveness of the research area, open issues and challenges are presented to spur future investigations, in which the involvement of edge cache, big data mining, social-aware device-to-device, cognitive radio, software defined network, and physical layer security for C-RANs are discussed, and the progress of testbed development and trial test are introduced as well.Comment: 27 pages, 11 figure

Feature Selection: A Data Perspective

Feature selection, as a data preprocessing strategy, has been proven to be effective and efficient in preparing data (especially high-dimensional data) for various data mining and machine learning problems. The objectives of feature selection include: building simpler and more comprehensible models, improving data mining performance, and preparing clean, understandable data. The recent proliferation of big data has presented some substantial challenges and opportunities to feature selection. In this survey, we provide a comprehensive and structured overview of recent advances in feature selection research. Motivated by current challenges and opportunities in the era of big data, we revisit feature selection research from a data perspective and review representative feature selection algorithms for conventional data, structured data, heterogeneous data and streaming data. Methodologically, to emphasize the differences and similarities of most existing feature selection algorithms for conventional data, we categorize them into four main groups: similarity based, information theoretical based, sparse learning based and statistical based methods. To facilitate and promote the research in this community, we also present an open-source feature selection repository that consists of most of the popular feature selection algorithms (\url{http://featureselection.asu.edu/}). Also, we use it as an example to show how to evaluate feature selection algorithms. At the end of the survey, we present a discussion about some open problems and challenges that require more attention in future research

Decomposition Principles and Online Learning in Cross-Layer Optimization for Delay-Sensitive Applications

In this paper, we propose a general cross-layer optimization framework in which we explicitly consider both the heterogeneous and dynamically changing characteristics of delay-sensitive applications and the underlying time-varying network conditions. We consider both the independently decodable data units (DUs, e.g. packets) and the interdependent DUs whose dependencies are captured by a directed acyclic graph (DAG). We first formulate the cross-layer design as a non-linear constrained optimization problem by assuming complete knowledge of the application characteristics and the underlying network conditions. The constrained cross-layer optimization is decomposed into several cross-layer optimization subproblems for each DU and two master problems. The proposed decomposition method determines the necessary message exchanges between layers for achieving the optimal cross-layer solution. However, the attributes (e.g. distortion impact, delay deadline etc) of future DUs as well as the network conditions are often unknown in the considered real-time applications. The impact of current cross-layer actions on the future DUs can be characterized by a state-value function in the Markov decision process (MDP) framework. Based on the dynamic programming solution to the MDP, we develop a low-complexity cross-layer optimization algorithm using online learning for each DU transmission. This online algorithm can be implemented in real-time in order to cope with unknown source characteristics, network dynamics and resource constraints. Our numerical results demonstrate the efficiency of the proposed online algorithm.Comment: 30 pages, 10 figure

Spatially Sparse Precoding in Millimeter Wave MIMO Systems

Millimeter wave (mmWave) signals experience orders-of-magnitude more pathloss than the microwave signals currently used in most wireless applications. MmWave systems must therefore leverage large antenna arrays, made possible by the decrease in wavelength, to combat pathloss with beamforming gain. Beamforming with multiple data streams, known as precoding, can be used to further improve mmWave spectral efficiency. Both beamforming and precoding are done digitally at baseband in traditional multi-antenna systems. The high cost and power consumption of mixed-signal devices in mmWave systems, however, make analog processing in the RF domain more attractive. This hardware limitation restricts the feasible set of precoders and combiners that can be applied by practical mmWave transceivers. In this paper, we consider transmit precoding and receiver combining in mmWave systems with large antenna arrays. We exploit the spatial structure of mmWave channels to formulate the precoding/combining problem as a sparse reconstruction problem. Using the principle of basis pursuit, we develop algorithms that accurately approximate optimal unconstrained precoders and combiners such that they can be implemented in low-cost RF hardware. We present numerical results on the performance of the proposed algorithms and show that they allow mmWave systems to approach their unconstrained performance limits, even when transceiver hardware constraints are considered.Comment: 30 pages, 7 figures, submitted to IEEE Transactions on Wireless Communication

Symbol-level and Multicast Precoding for Multiuser Multiantenna Downlink: A Survey, Classification and Challenges

Precoding has been conventionally considered as an effective means of mitigating the interference and efficiently exploiting the available in the multiantenna downlink channel, where multiple users are simultaneously served with independent information over the same channel resources. The early works in this area were focused on transmitting an individual information stream to each user by constructing weighted linear combinations of symbol blocks (codewords). However, more recent works have moved beyond this traditional view by: i) transmitting distinct data streams to groups of users and ii) applying precoding on a symbol-per-symbol basis. In this context, the current survey presents a unified view and classification of precoding techniques with respect to two main axes: i) the switching rate of the precoding weights, leading to the classes of block- and symbol-level precoding, ii) the number of users that each stream is addressed to, hence unicast-/multicast-/broadcast- precoding. Furthermore, the classified techniques are compared through representative numerical results to demonstrate their relative performance and uncover fundamental insights. Finally, a list of open theoretical problems and practical challenges are presented to inspire further research in this area.Comment: Submitted to IEEE Communications Surveys & Tutorial

Load Balancing User Association in Millimeter Wave MIMO Networks

User association is necessary in dense millimeter wave (mmWave) networks to determine which base station a user connects to in order to balance base station loads and maximize throughput. Given that mmWave connections are highly directional and vulnerable to small channel variations, user association changes these connections and hence significantly affects the user's instantaneous rate as well as network interference. In this paper, we introduce a new load balancing user association scheme for mmWave MIMO networks which considers this dependency on user association of user's transmission rates and network interference. We formulate the user association problem as mixed integer nonlinear programming and design a polynomial-time algorithm, called Worst Connection Swapping (WCS), to find a near-optimal solution. Simulation results confirm that the proposed user association scheme improves network performance significantly by moving the traffic of congested base stations to lightly-loaded ones and adjusting the interference accordingly. Further, the proposed WCS algorithm outperforms other generic algorithms for combinatorial programming such as the genetic algorithm in both accuracy and speed at several orders of magnitude faster, and for small networks where exhaustive search is possible it reaches the optimal solution.Comment: 15 pages, 8 figures, Submitted to IEEE Transactions on Wireless Communication

On the Packet Allocation of Multi-Band Aggregation Wireless Networks

The use of heterogeneous networks with multiple radio access technologies (RATs) is a system concept that both academia and industry are studying. In such system, integrated use of available multiple RATs is essential to achieve beyond additive throughput and connectivity gains using multi-dimensional diversity. This paper considers an aggregation module called opportunistic multi-MAC aggregation (OMMA). It resides between the IP layer and the air interface protocol stacks, common to all RATs in the device. We present a theoretical framework for such system while considering a special case of multi-RAT systems, i.e., a multi-band wireless LAN (WLAN) system. An optimal packet distribution approach is derived which minimizes the average packet latency (the sum of queueing delay and serving delay) over multiple bands. It supports multiple user terminals with different QoS classes simultaneously. We further propose a packet scheduling algorithm, OMMA Leaky Bucket, which minimizes the packet end-to-end delay, i.e., the sum of average packet latency and average packet reordering delay. We also describe the system architecture of the proposed OMMA system, which is applicable for the general case of the multi- RAT devices. It includes functional description, discovery and association processes, and dynamic RAT update management. We finally present simulation results for a multi-band WLAN system. It shows the performance gains of the proposed OMMA Leaky Bucket scheme in comparison to other existing packet scheduling mechanisms.Comment: The final publication is available at Springer via https://link.springer.com/article/10.1007/s11276-017-1486-