5,167 research outputs found
Diffusion Adaptation Strategies for Distributed Estimation over Gaussian Markov Random Fields
The aim of this paper is to propose diffusion strategies for distributed
estimation over adaptive networks, assuming the presence of spatially
correlated measurements distributed according to a Gaussian Markov random field
(GMRF) model. The proposed methods incorporate prior information about the
statistical dependency among observations, while at the same time processing
data in real-time and in a fully decentralized manner. A detailed mean-square
analysis is carried out in order to prove stability and evaluate the
steady-state performance of the proposed strategies. Finally, we also
illustrate how the proposed techniques can be easily extended in order to
incorporate thresholding operators for sparsity recovery applications.
Numerical results show the potential advantages of using such techniques for
distributed learning in adaptive networks deployed over GMRF.Comment: Submitted to IEEE Transactions on Signal Processing. arXiv admin
note: text overlap with arXiv:1206.309
Stochastic Training of Neural Networks via Successive Convex Approximations
This paper proposes a new family of algorithms for training neural networks
(NNs). These are based on recent developments in the field of non-convex
optimization, going under the general name of successive convex approximation
(SCA) techniques. The basic idea is to iteratively replace the original
(non-convex, highly dimensional) learning problem with a sequence of (strongly
convex) approximations, which are both accurate and simple to optimize.
Differently from similar ideas (e.g., quasi-Newton algorithms), the
approximations can be constructed using only first-order information of the
neural network function, in a stochastic fashion, while exploiting the overall
structure of the learning problem for a faster convergence. We discuss several
use cases, based on different choices for the loss function (e.g., squared loss
and cross-entropy loss), and for the regularization of the NN's weights. We
experiment on several medium-sized benchmark problems, and on a large-scale
dataset involving simulated physical data. The results show how the algorithm
outperforms state-of-the-art techniques, providing faster convergence to a
better minimum. Additionally, we show how the algorithm can be easily
parallelized over multiple computational units without hindering its
performance. In particular, each computational unit can optimize a tailored
surrogate function defined on a randomly assigned subset of the input
variables, whose dimension can be selected depending entirely on the available
computational power.Comment: Preprint submitted to IEEE Transactions on Neural Networks and
Learning System
Distributed Estimation and Control of Algebraic Connectivity over Random Graphs
In this paper we propose a distributed algorithm for the estimation and
control of the connectivity of ad-hoc networks in the presence of a random
topology. First, given a generic random graph, we introduce a novel stochastic
power iteration method that allows each node to estimate and track the
algebraic connectivity of the underlying expected graph. Using results from
stochastic approximation theory, we prove that the proposed method converges
almost surely (a.s.) to the desired value of connectivity even in the presence
of imperfect communication scenarios. The estimation strategy is then used as a
basic tool to adapt the power transmitted by each node of a wireless network,
in order to maximize the network connectivity in the presence of realistic
Medium Access Control (MAC) protocols or simply to drive the connectivity
toward a desired target value. Numerical results corroborate our theoretical
findings, thus illustrating the main features of the algorithm and its
robustness to fluctuations of the network graph due to the presence of random
link failures.Comment: To appear in IEEE Transactions on Signal Processin
Sparse Distributed Learning Based on Diffusion Adaptation
This article proposes diffusion LMS strategies for distributed estimation
over adaptive networks that are able to exploit sparsity in the underlying
system model. The approach relies on convex regularization, common in
compressive sensing, to enhance the detection of sparsity via a diffusive
process over the network. The resulting algorithms endow networks with learning
abilities and allow them to learn the sparse structure from the incoming data
in real-time, and also to track variations in the sparsity of the model. We
provide convergence and mean-square performance analysis of the proposed method
and show under what conditions it outperforms the unregularized diffusion
version. We also show how to adaptively select the regularization parameter.
Simulation results illustrate the advantage of the proposed filters for sparse
data recovery.Comment: to appear in IEEE Trans. on Signal Processing, 201
Distributed Detection and Estimation in Wireless Sensor Networks
In this article we consider the problems of distributed detection and
estimation in wireless sensor networks. In the first part, we provide a general
framework aimed to show how an efficient design of a sensor network requires a
joint organization of in-network processing and communication. Then, we recall
the basic features of consensus algorithm, which is a basic tool to reach
globally optimal decisions through a distributed approach. The main part of the
paper starts addressing the distributed estimation problem. We show first an
entirely decentralized approach, where observations and estimations are
performed without the intervention of a fusion center. Then, we consider the
case where the estimation is performed at a fusion center, showing how to
allocate quantization bits and transmit powers in the links between the nodes
and the fusion center, in order to accommodate the requirement on the maximum
estimation variance, under a constraint on the global transmit power. We extend
the approach to the detection problem. Also in this case, we consider the
distributed approach, where every node can achieve a globally optimal decision,
and the case where the decision is taken at a central node. In the latter case,
we show how to allocate coding bits and transmit power in order to maximize the
detection probability, under constraints on the false alarm rate and the global
transmit power. Then, we generalize consensus algorithms illustrating a
distributed procedure that converges to the projection of the observation
vector onto a signal subspace. We then address the issue of energy consumption
in sensor networks, thus showing how to optimize the network topology in order
to minimize the energy necessary to achieve a global consensus. Finally, we
address the problem of matching the topology of the network to the graph
describing the statistical dependencies among the observed variables.Comment: 92 pages, 24 figures. To appear in E-Reference Signal Processing, R.
Chellapa and S. Theodoridis, Eds., Elsevier, 201
Signals on Graphs: Uncertainty Principle and Sampling
In many applications, the observations can be represented as a signal defined
over the vertices of a graph. The analysis of such signals requires the
extension of standard signal processing tools. In this work, first, we provide
a class of graph signals that are maximally concentrated on the graph domain
and on its dual. Then, building on this framework, we derive an uncertainty
principle for graph signals and illustrate the conditions for the recovery of
band-limited signals from a subset of samples. We show an interesting link
between uncertainty principle and sampling and propose alternative signal
recovery algorithms, including a generalization to frame-based reconstruction
methods. After showing that the performance of signal recovery algorithms is
significantly affected by the location of samples, we suggest and compare a few
alternative sampling strategies. Finally, we provide the conditions for perfect
recovery of a useful signal corrupted by sparse noise, showing that this
problem is also intrinsically related to vertex-frequency localization
properties.Comment: This article is the revised version submitted to the IEEE
Transactions on Signal Processing on May, 2016; first revision was submitted
on January, 2016; original manuscript was submitted on July, 2015. The work
includes 16 pages, 8 figure
Adaptive Graph Signal Processing: Algorithms and Optimal Sampling Strategies
The goal of this paper is to propose novel strategies for adaptive learning
of signals defined over graphs, which are observed over a (randomly
time-varying) subset of vertices. We recast two classical adaptive algorithms
in the graph signal processing framework, namely, the least mean squares (LMS)
and the recursive least squares (RLS) adaptive estimation strategies. For both
methods, a detailed mean-square analysis illustrates the effect of random
sampling on the adaptive reconstruction capability and the steady-state
performance. Then, several probabilistic sampling strategies are proposed to
design the sampling probability at each node in the graph, with the aim of
optimizing the tradeoff between steady-state performance, graph sampling rate,
and convergence rate of the adaptive algorithms. Finally, a distributed RLS
strategy is derived and is shown to be convergent to its centralized
counterpart. Numerical simulations carried out over both synthetic and real
data illustrate the good performance of the proposed sampling and
reconstruction strategies for (possibly distributed) adaptive learning of
signals defined over graphs.Comment: Submitted to IEEE Transactions on Signal Processing, September 201
Bilingual Schooling and Earnings: Evidence from a Language-in-Education reform [WP]
We exploit the 1983 language-in-education reform that introduced Catalan alongside Spanish as medium of instruction in Catalan schools to estimate the labour market value of bilingual education. Identification is achieved in a difference-in-differences framework exploiting variation in exposure to the reform across years of schooling and years of birth. We find positive wage returns to bilingual education and no effects on employment, hours of work or occupation. Results are robust to education-cohort specific trends or selection into schooling and are mainly stemming from exposure at compulsory education. We show that the effect worked through increased Catalan proficiency for Spanish speakers and that there were also positive effects for Catalan speakers from families with low education. These findings are consistent with human capital effects rather than with more efficient job search or reduced discrimination. Exploiting the heterogeneous effects of the reform as an instrument for proficiency we find sizeable earnings effects of skills in Catalan
- …