A Sparsity-Aware Adaptive Algorithm for Distributed Learning

In this paper, a sparsity-aware adaptive algorithm for distributed learning in diffusion networks is developed. The algorithm follows the set-theoretic estimation rationale. At each time instance and at each node of the network, a closed convex set, known as property set, is constructed based on the received measurements; this defines the region in which the solution is searched for. In this paper, the property sets take the form of hyperslabs. The goal is to find a point that belongs to the intersection of these hyperslabs. To this end, sparsity encouraging variable metric projections onto the hyperslabs have been adopted. Moreover, sparsity is also imposed by employing variable metric projections onto weighted $\ell_1$ balls. A combine adapt cooperation strategy is adopted. Under some mild assumptions, the scheme enjoys monotonicity, asymptotic optimality and strong convergence to a point that lies in the consensus subspace. Finally, numerical examples verify the validity of the proposed scheme, compared to other algorithms, which have been developed in the context of sparse adaptive learning

Sparse Distributed Learning Based on Diffusion Adaptation

This article proposes diffusion LMS strategies for distributed estimation over adaptive networks that are able to exploit sparsity in the underlying system model. The approach relies on convex regularization, common in compressive sensing, to enhance the detection of sparsity via a diffusive process over the network. The resulting algorithms endow networks with learning abilities and allow them to learn the sparse structure from the incoming data in real-time, and also to track variations in the sparsity of the model. We provide convergence and mean-square performance analysis of the proposed method and show under what conditions it outperforms the unregularized diffusion version. We also show how to adaptively select the regularization parameter. Simulation results illustrate the advantage of the proposed filters for sparse data recovery.Comment: to appear in IEEE Trans. on Signal Processing, 201

Trust-Based Distributed Kalman Filtering for Target Tracking under Malicious Cyber Attacks

As one of the widely used applications in wireless sensor networks, target tracking has attracted considerable attention. Although many tracking techniques have been developed, it is still a challenging problem if the network is under cyber attacks. Inaccurate or false information is maliciously broadcast by the compromised nodes to their neighbors. They are likely to threaten the security of the system and result in performance deterioration. In this paper, a distributed Kalman filtering technique with trust-based dynamic combination strategy is developed to improve resilience against cyber attacks. Furthermore, it is efficient in terms of communication load, only local instantaneous estimates are exchanged with the neighboring nodes. Numerical results are provided to evaluate the performance of the proposed approach by considering random, false data injection and replay attacks

Trading off Complexity With Communication Costs in Distributed Adaptive Learning via Krylov Subspaces for Dimensionality Reduction

In this paper, the problemof dimensionality reduction in adaptive distributed learning is studied. We consider a network obeying the ad-hoc topology, in which the nodes sense an amount of data and cooperate with each other, by exchanging information, in order to estimate an unknown, common, parameter vector. The algorithm, to be presented here, follows the set-theoretic estimation rationale; i.e., at each time instant and at each node of the network, a closed convex set is constructed based on the received measurements, and this defines the region in which the solution is searched for. In this paper, these closed convex sets, known as property sets, take the form of hyperslabs. Moreover, in order to reduce the number of transmitted coefficients, which is dictated by the dimension of the unknown vector, we seek for possible solutions in a subspace of lower dimension; the technique will be developed around the Krylov subspace rationale. Our goal is to find a point that belongs to the intersection of this infinite number of hyperslabs and the respective Krylov subspaces. This is achieved via a sequence of projections onto the property sets and the Krylov subspaces. The case of highly correlated inputs that degrades the performance of the algorithm is also considered. This is overcome via a transformation whichwhitens the input. The proposed schemes are brought in a decentralized form by adopting the combine-adapt cooperation strategy among the nodes. Full convergence analysis is carried out and numerical tests verify the validity of the proposed schemes in different scenarios in the context of the adaptive distributed system identification task

Aeronautical engineering: A continuing bibliography with indexes (supplement 272)

This bibliography lists 719 reports, articles, and other documents introduced into the NASA scientific and technical information system in November, 1991. Subject coverage includes: design, construction and testing of aircraft and aircraft engines; aircraft components, equipment, and systems; ground support systems; and theoretical and applied aspects of aerodynamics and general fluid dynamics

Energy Efficient High Port Count Optical Switches

The advance of internet applications, such as video streaming, big data and cloud computing, is reshaping the telecommunication and internet industries. Bandwidth demands in datacentres have been boosted by these emerging data-hungry internet applications. Regarding inter- and intra-datacentre communications, fine-grained data need to be exchanged across a large shared memory space. Large-scale high-speed optical switches tend to use a rearrangeably non-blocking architecture as this limits the number of switching elements required. However, this comes at the expense of requiring more sophisticated route selection within the switch and also some forms of time-slotted protocols. The looping algorithm is the classical routing algorithm to set up paths in rearrangeably non-blocking switches. It was born in the electronic switch era, where all links in the switches are equal. It is, therefore, not able to accommodate loss difference between optical paths due to the different length of waveguides and distinct numbers of crossings, and bends, leading to sub-optimal performance. We, therefore, propose an advanced path-selection algorithm based on the looping algorithm that minimises the path-dependent loss. It explores all possible set-ups for a given connection assignment and selects the optimal one. It guarantees that no individual path would have a sufficiently substantial loss, therefore, improve the overall performance of the switch. The performance of the proposed algorithm has been assessed by modelling switches using the VPI simulator. An 8×8 Clos-tree switch demonstrates a 2.7dB decrease in loss and 1.9dB improvement in IPDR with 1.5 dB penalty for the worst case. An 8×8 dilated Beneš shows more than 4 dB loss reduction for the lossiest path and 1.4 dB IPDR improvement for 1 dB power penalty. The improved algorithm can be run once for each switch design and store its output in a compact lookup table, enabling rapid switch reconfiguration. Microelectromechanical systems (MEMS) based optical switches have been fabricated with over 1,000 ports which meet the port count requirements in data centre networks. However, the reconfiguration speed of the MEMS switches is limited to the millisecond to microsecond timescale, which is not sufficient for packet switching in datacentres. Opto-electronic devices, such as Mach-Zehnder Interferometers (MZIs) and semiconductor optical amplifiers (SOAs) with nanosecond response time show the potential to fulfil the requirements of packet switching. However, the scalability of MZI switches is inherently limited by insertion loss and accumulated crosstalk, while the scalability of SOA switches is restricted by accumulated noise and distortion. We, therefore, have proposed a dilated Beneš hybrid MZI-SOA design, where MZIs are implemented as 1×2 or 2×1 low-loss switching elements, minimising crosstalk by using a single input, and where short SOAs are included as gain or absorption units, offering either loss compensation or crosstalk suppression though adding only minimal noise and distortion. A 4×4 device has been fabricated and exhibits a mere 1.3dB loss, an extinction ratio of 47dB, and more than 13dB IPDR for a 0.5dB power penalty. When operating with 10 Gb/s per port, 6pJ/bit energy consumption is demonstrated, delivering 20% reduced energy consumption compared with SOA-based switches. The tolerance of the current control accuracy of this switch is very broad. Within a 5 mA bias current range, the power penalty can be maintained below 0.2 dB for 8 dB IPDR and 12 mA for 10 dB IPDR with a penalty less 0.5 dB. The excellent crosstalk and power penalty performance demonstrated by this chip enable the scalability of this hybrid approach. The performance of 16×16 port dilated Beneš hybrid switch is experimentally assessed by cascading 4×4 switch chips, demonstrating an IPDR of 15 dB at a 1 dB penalty with a 0.6 dB power penalty floor. In terms of switches with port count larger than 16×16, the power penalty performance has been analysed with physical layer simulations fitted with state-of-the-art data. We assess the feasibility of three potential topologies, with different architectural optimisations: dilated Beneš, Beneš and Clos-Beneš. Quantitative analysis for switches with up to 2048 ports is presented, achieving a 1.15dB penalty for a BER of 10-3, compatible with soft-decision forward error correction.Cambridge Overseas Trust; China Scholarship Council

Distributed detection and estimation in wireless sensor networks: resource allocation, fusion rules, and network security

This thesis addresses the problem of detection of an unknown binary event. In particular, we consider centralized detection, distributed detection, and network security in wireless sensor networks (WSNs). The communication links among SNs are subject to limited SN transmit power, limited bandwidth (BW), and are modeled as orthogonal channels with path loss, flat fading and additive white Gaussian noise (AWGN). We propose algorithms for resource allocations, fusion rules, and network security. In the first part of this thesis, we consider the centralized detection and calculate the optimal transmit power allocation and the optimal number of quantization bits for each SN. The resource allocation is performed at the fusion center (FC) and it is referred as a centralized approach. We also propose a novel fully $distributed$ algorithm to address this resource allocation problem. What makes this scheme attractive is that the SNs share with their neighbors just their individual transmit power at the current states. Finally, the optimal soft fusion rule at the FC is derived. But as this rule requires a-priori knowledge that is difficult to attain in practice, suboptimal fusion rules are proposed that are realizable in practice. The second part considers a fully distributed detection framework and we propose a two-step distributed quantized fusion rule algorithm where in the first step the SNs collaborate with their neighbors through error-free, orthogonal channels. In the second step, local 1-bit decisions generated in the first step are shared among neighbors to yield a consensus. A binary hypothesis testing is performed at any arbitrary SN to optimally declare the global decision. Simulations show that our proposed quantized two-step distributed detection algorithm approaches the performance of the unquantized centralized (with a FC) detector and its power consumption is shown to be 50% less than the existing (unquantized) conventional algorithm. Finally, we analyze the detection performance of under-attack WSNs and derive attacking and defense strategies from both the Attacker and the FC perspective. We re-cast the problem as a minimax game between the FC and Attacker and show that the Nash Equilibrium (NE) exists. We also propose a new non-complex and efficient reputation-based scheme to identify these compromised SNs. Based on this reputation metric, we propose a novel FC weight computation strategy ensuring that the weights for the identified compromised SNs are likely to be decreased. In this way, the FC decides how much a SN should contribute to its final decision. We show that this strategy outperforms the existing schemes

Adaptive sequential optimization with applications to machine learning

The focus of this thesis is on solving a sequence of optimization problems that change over time in a structured manner. This type of problem naturally arises in contexts as diverse as channel estimation, target tracking, sequential machine learning, and repeated games. Due to the time-varying nature of these problems, it is necessary to determine new solutions as the problems change in order to ensure good solution quality. However, since the problems change over time in a structured manner, it is beneficial to exploit solutions to the previous optimization problems in order to efficiently solve the current optimization problem. The first problem considered is sequentially solving minimization problems that change slowly, in the sense that the gap between successive minimizers is bounded in norm. The minimization problems are solved by sequentially applying a selected optimization algorithm, such as stochastic gradient descent (SGD), based on drawing a number of samples in order to carry out a desired number of iterations. Two tracking criteria are introduced to evaluate approximate minimizer quality: one based on being accurate with respect to the mean trajectory, and the other based on being accurate in high probability (IHP). Knowledge of the bound on how the minimizers change, combined with properties of the chosen optimization algorithm, is used to select the number of samples needed to meet the desired tracking criterion. Next, it is not assumed that the bound on how the minimizers change is known. A technique to estimate the change in minimizers is provided along with analysis to show that eventually the estimate upper bounds the change in minimizers. This estimate of the change in minimizers is combined with the previous analysis to provide sample size selection rules to ensure that the mean or IHP tracking criterion is met. Simulations are used to confirm that the estimation approach provides the desired tracking accuracy in practice. An application of this framework to machine learning problems is considered next. A cost-based approach is introduced to select the number of samples with a cost function for taking a number of samples and a cost budget over a fixed horizon. An extension of this framework is developed to apply cross validation for model selection. Finally, experiments with synthetic and real data are used to confirm that this approach performs well for machine learning problems. The next model considered is solving a sequence of minimization problems with the possibility that there can be abrupt jumps in the minimizers mixed in with the normal slow changes. Alternative approaches are introduced to estimate the changes in the minimizers and select the number of samples. A simulation experiment demonstrates the effectiveness of this approach. Finally, a variant of this framework is applied to learning in games. A sequence of repeated games is considered in which the underlying stage games themselves vary slowly over time in the sense that the pure strategy Nash equilibria change slowly. Approximate pure-strategy Nash equilibria are learned for this sequence of zero sum games. A technique is introduced to estimate the change in the Nash equilibiria as for the sequence of minimization problems. Applications to a synthetic game and a game based on a surveillance network problem are introduced to demonstrate the game framework