Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition

Dynamic mode decomposition (DMD) provides a practical means of extracting insightful dynamical information from fluids datasets. Like any data processing technique, DMD's usefulness is limited by its ability to extract real and accurate dynamical features from noise-corrupted data. Here we show analytically that DMD is biased to sensor noise, and quantify how this bias depends on the size and noise level of the data. We present three modifications to DMD that can be used to remove this bias: (i) a direct correction of the identified bias using known noise properties, (ii) combining the results of performing DMD forwards and backwards in time, and (iii) a total least-squares-inspired algorithm. We discuss the relative merits of each algorithm, and demonstrate the performance of these modifications on a range of synthetic, numerical, and experimental datasets. We further compare our modified DMD algorithms with other variants proposed in recent literature

Sparsity-Based Error Detection in DC Power Flow State Estimation

This paper presents a new approach for identifying the measurement error in the DC power flow state estimation problem. The proposed algorithm exploits the singularity of the impedance matrix and the sparsity of the error vector by posing the DC power flow problem as a sparse vector recovery problem that leverages the structure of the power system and uses $l_1$-norm minimization for state estimation. This approach can provably compute the measurement errors exactly, and its performance is robust to the arbitrary magnitudes of the measurement errors. Hence, the proposed approach can detect the noisy elements if the measurements are contaminated with additive white Gaussian noise plus sparse noise with large magnitude. The effectiveness of the proposed sparsity-based decomposition-DC power flow approach is demonstrated on the IEEE 118-bus and 300-bus test systems

Estimation of Autoregressive Parameters from Noisy Observations Using Iterated Covariance Updates

Estimating the parameters of the autoregressive (AR) random process is a problem that has been well-studied. In many applications, only noisy measurements of AR process are available. The effect of the additive noise is that the system can be modeled as an AR model with colored noise, even when the measurement noise is white, where the correlation matrix depends on the AR parameters. Because of the correlation, it is expedient to compute using multiple stacked observations. Performing a weighted least-squares estimation of the AR parameters using an inverse covariance weighting can provide significantly better parameter estimates, with improvement increasing with the stack depth. The estimation algorithm is essentially a vector RLS adaptive filter, with time-varying covariance matrix. Different ways of estimating the unknown covariance are presented, as well as a method to estimate the variances of the AR and observation noise. The notation is extended to vector autoregressive (VAR) processes. Simulation results demonstrate performance improvements in coefficient error and in spectrum estimation