Sparse Graphical Linear Dynamical Systems

Time-series datasets are central in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Estimating the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the static graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on synthetic and real weather variability data showcases the effectiveness of the proposed model and inference algorithm

Certifying the Optimality of a Distributed State Estimation System via Majorization Theory

Consider a first order linear time-invariant discrete time system driven by process noise, a pre-processor that accepts causal measurements of the state of the system, and a state estimator. The pre-processor and the state estimator are not co-located, and, at every time-step, the pre-processor transmits either a real number or an erasure symbol to the estimator. We seek the pre-processor and the estimator that jointly minimize a cost that combines two terms; the expected squared state estimation error and a communication cost. In our formulation, the transmission of a real number from the pre-processor to the estimator incurs a positive cost while erasures induce zero cost. This paper is the first to prove analytically that a symmetric threshold policy at the pre-processor and a Kalman-like filter at the estimator, which updates its estimate linearly in the presence of erasures, are jointly optimal for our problem

DISTRIBUTED ESTIMATION OVER NETWORKS WITH COMMUNICATION COSTS

We analyze how distributed or decentralized estimation can be performed over networks, when there is a price to be paid whenever nodes in the network communicate with each other. The work here has application especially in the network control systems. Assume that different nodes in the network can track perfectly or with imperfectly some stochastic processes, while other nodes in the network need to estimate these stochastic processes. The nodes which can observe the stochastic processes can send information directly to the nodes which need to estimate the processes, or information can be sent to intermediate nodes. When each transmission is performed a cost for communication is paid. The goal of the network is to optimize jointly a cost which consists both of a function of the estimation error and a function of the transmission cost. We show here that for some simple topologies the decision to send information over the network is a threshold policy, while the estimators are linear estimators which resemble with the Kalman-filter. For the result dealing with simple topologies we have proved the results using majorization theory. It is also shown here both analytically and numerically that things can immediately become quite complicated. If we take into consideration multidimensional problems or problems with multiple agents and/or transmission noise, the optimal strategies can no longer be found analytically and it can be quite difficult to compute numerically the optimal strategies

To Drop or Not to Drop: Receiver Design Principles for Estimation over Wireless Links

In this paper we consider estimation of a multiple-input multiple-output dynamical system over a wireless fading communication channel using a Kalman filter. We are interested in finding the optimum receiver design in terms of handing noisy samples. We reformulate the estimation problem to include the impact of stochastic communication noise in the noisy packets. We will show how the eigenvalues of the state transition matrix A affect the optimum receiver design. We prove that, in the absence of a cross-layer information path, packet drop should be designed to balance information loss and communication noise in order to optimize the performance. In the presence of a cross-layer path, we show that keeping all the packets will minimize the average estimation error covariance. We also derive the stability condition in the presence of noisy packets and prove that it is independent of the shape of the communication noise variance or availability of a cross-layer information path