Distributed Recursive Least-Squares: Stability and Performance Analysis

The recursive least-squares (RLS) algorithm has well-documented merits for reducing complexity and storage requirements, when it comes to online estimation of stationary signals as well as for tracking slowly-varying nonstationary processes. In this paper, a distributed recursive least-squares (D-RLS) algorithm is developed for cooperative estimation using ad hoc wireless sensor networks. Distributed iterations are obtained by minimizing a separable reformulation of the exponentially-weighted least-squares cost, using the alternating-minimization algorithm. Sensors carry out reduced-complexity tasks locally, and exchange messages with one-hop neighbors to consent on the network-wide estimates adaptively. A steady-state mean-square error (MSE) performance analysis of D-RLS is conducted, by studying a stochastically-driven `averaged' system that approximates the D-RLS dynamics asymptotically in time. For sensor observations that are linearly related to the time-invariant parameter vector sought, the simplifying independence setting assumptions facilitate deriving accurate closed-form expressions for the MSE steady-state values. The problems of mean- and MSE-sense stability of D-RLS are also investigated, and easily-checkable sufficient conditions are derived under which a steady-state is attained. Without resorting to diminishing step-sizes which compromise the tracking ability of D-RLS, stability ensures that per sensor estimates hover inside a ball of finite radius centered at the true parameter vector, with high-probability, even when inter-sensor communication links are noisy. Interestingly, computer simulations demonstrate that the theoretical findings are accurate also in the pragmatic settings whereby sensors acquire temporally-correlated data.Comment: 30 pages, 4 figures, submitted to IEEE Transactions on Signal Processin

Diffusion recursive least squares algorithm based on triangular decomposition

In this paper, diffusion strategies used by QR-decomposition based on recursive least squares algorithm (DQR-RLS) and the sign version of DQR-RLS algorithm (DQR-sRLS) are introduced for distributed networks. In terms of the QR-decomposition method and Cholesky factorization, a modified Kalman vector is given adaptively with the help of unitary rotation that can decrease the complexity from inverse autocorrelation matrix to vector. According to the diffusion strategies, combine-then-adapt (CTA) and adapt-then-combine (ATC) based on DQR-RLS and DQR-sRLS algorithms are proposed with the combination and adaptation steps. To minimize the cost function, diffused versions of CTA-DQR-RLS, ATC-DQR-RLS, CTA-DQR-sRLS and ATC-DiQR-sRLS algorithms are compared. Simulation results depict that the proposed DQR-RLS-based and DQR-sRLS-based algorithms can clearly achieve the better performance than the standard combine-then-adapt-diffusion RLS (CTA-DRLS) and ATC-DRLS mechanisms

Learning and Prediction Theory of Distributed Least Squares

With the fast development of the sensor and network technology, distributed estimation has attracted more and more attention, due to its capability in securing communication, in sustaining scalability, and in enhancing safety and privacy. In this paper, we consider a least-squares (LS)-based distributed algorithm build on a sensor network to estimate an unknown parameter vector of a dynamical system, where each sensor in the network has partial information only but is allowed to communicate with its neighbors. Our main task is to generalize the well-known theoretical results on the traditional LS to the current distributed case by establishing both the upper bound of the accumulated regrets of the adaptive predictor and the convergence of the distributed LS estimator, with the following key features compared with the existing literature on distributed estimation: Firstly, our theory does not need the previously imposed independence, stationarity or Gaussian property on the system signals, and hence is applicable to stochastic systems with feedback control. Secondly, the cooperative excitation condition introduced and used in this paper for the convergence of the distributed LS estimate is the weakest possible one, which shows that even if any individual sensor cannot estimate the unknown parameter by the traditional LS, the whole network can still fulfill the estimation task by the distributed LS. Moreover, our theoretical analysis is also different from the existing ones for distributed LS, because it is an integration of several powerful techniques including stochastic Lyapunov functions, martingale convergence theorems, and some inequalities on convex combination of nonnegative definite matrices.Comment: 14 pages, submitted to IEEE Transactions on Automatic Contro

Multitask Diffusion Adaptation over Networks

Adaptive networks are suitable for decentralized inference tasks, e.g., to monitor complex natural phenomena. Recent research works have intensively studied distributed optimization problems in the case where the nodes have to estimate a single optimum parameter vector collaboratively. However, there are many important applications that are multitask-oriented in the sense that there are multiple optimum parameter vectors to be inferred simultaneously, in a collaborative manner, over the area covered by the network. In this paper, we employ diffusion strategies to develop distributed algorithms that address multitask problems by minimizing an appropriate mean-square error criterion with $\ell_2$-regularization. The stability and convergence of the algorithm in the mean and in the mean-square sense is analyzed. Simulations are conducted to verify the theoretical findings, and to illustrate how the distributed strategy can be used in several useful applications related to spectral sensing, target localization, and hyperspectral data unmixing.Comment: 29 pages, 11 figures, submitted for publicatio

Online estimation of battery equivalent circuit model parameters and state of charge using decoupled least squares technique

Battery equivalent circuit models (ECMs) are widely employed in online battery management applications. The model parameters are known to vary according to the operating conditions, such as the battery state of charge (SOC) and the ambient temperature. Therefore, online recursive ECM parameter estimation is one means that may help to improve the modelling accuracy. Because a battery system consists of both fast and slow dynamics, the classical least squares (LS) method, that estimates together all the model parameters, is known to suffer from numerical problems and poor accuracy. The aim of this paper is to overcome this problem by proposing a new decoupled weighted recursive least squares (DWRLS) method, which estimates separately the parameters of the battery fast and slow dynamics. Another issue is that, the ECM-based SOC estimator generally requires a full-order state observer, which will increase the algorithm’s complexity and the time required for the filter tuning. In this work, the battery SOC estimation is achieved based on the parameter estimation results. This circumvents the additional full-order observer, leading to a reduced complexity. An extensive simulation study is conducted to compare the proposed method against the traditional LS technique. The proposed approach is also applied to estimate the parameters of ECM where the experimental data are collected using a cylindrical 3Ah 18650-type Li ion NCA cell. Finally, both the simulation and experimental results in this study have demonstrated that the proposed DWRLS approach can improve not only the modelling accuracy but also the SOC estimation performance compared with the LS algorithm

Distributed Least Squares Algorithm for Continuous-time Stochastic Systems Under Cooperative Excitation Condition

In this paper, we study the distributed adaptive estimation problem of continuous-time stochastic dynamic systems over sensor networks where each agent can only communicate with its local neighbors. A distributed least squares (LS) algorithm based on diffusion strategy is proposed such that the sensors can cooperatively estimate the unknown time-invariant parameter vector from continuous-time noisy signals. By using the martingal estimation theory and Ito formula, we provide upper bounds for the estimation error of the proposed distributed LS algorithm, and further obtain the convergence results under a cooperative excitation condition. Compared with the existing results, our results are established without using the boundedness or persistent excitation (PE) conditions of regression signals. We provide simulation examples to show that multiple sensors can cooperatively accomplish the estimation task even if any individual can not