39 research outputs found
Load curve data cleansing and imputation via sparsity and low rank
The smart grid vision is to build an intelligent power network with an
unprecedented level of situational awareness and controllability over its
services and infrastructure. This paper advocates statistical inference methods
to robustify power monitoring tasks against the outlier effects owing to faulty
readings and malicious attacks, as well as against missing data due to privacy
concerns and communication errors. In this context, a novel load cleansing and
imputation scheme is developed leveraging the low intrinsic-dimensionality of
spatiotemporal load profiles and the sparse nature of "bad data.'' A robust
estimator based on principal components pursuit (PCP) is adopted, which effects
a twofold sparsity-promoting regularization through an -norm of the
outliers, and the nuclear norm of the nominal load profiles. Upon recasting the
non-separable nuclear norm into a form amenable to decentralized optimization,
a distributed (D-) PCP algorithm is developed to carry out the imputation and
cleansing tasks using networked devices comprising the so-termed advanced
metering infrastructure. If D-PCP converges and a qualification inequality is
satisfied, the novel distributed estimator provably attains the performance of
its centralized PCP counterpart, which has access to all networkwide data.
Computer simulations and tests with real load curve data corroborate the
convergence and effectiveness of the novel D-PCP algorithm.Comment: 8 figures, submitted to IEEE Transactions on Smart Grid - Special
issue on "Optimization methods and algorithms applied to smart grid
Matrix Completion with Noise via Leveraged Sampling
Many matrix completion methods assume that the data follows the uniform
distribution. To address the limitation of this assumption, Chen et al.
\cite{Chen20152999} propose to recover the matrix where the data follows the
specific biased distribution. Unfortunately, in most real-world applications,
the recovery of a data matrix appears to be incomplete, and perhaps even
corrupted information. This paper considers the recovery of a low-rank matrix,
where some observed entries are sampled in a \emph{biased distribution}
suitably dependent on \emph{leverage scores} of a matrix, and some observed
entries are uniformly corrupted. Our theoretical findings show that we can
provably recover an unknown matrix of rank from just about
entries even when the few observed entries are corrupted with a
small amount of noisy information. Empirical studies verify our theoretical
results
Motion Capture Data Completion via Truncated Nuclear Norm Regularization
The objective of motion capture (mocap) data completion is to recover missing measurement of the body markers from mocap. It becomes increasingly challenging as the missing ratio and duration of mocap data grow. Traditional approaches usually recast this problem as a low-rank matrix approximation problem based on the nuclear norm. However, the nuclear norm defined as the sum of all the singular values of a matrix is not a good approximation to the rank of mocap data. This paper proposes a novel approach to solve mocap data completion problem by adopting a new matrix norm, called truncated nuclear norm. An efficient iterative algorithm is designed to solve this problem based on the augmented Lagrange multiplier. The convergence of the proposed method is proved mathematically under mild conditions. To demonstrate the effectiveness of the proposed method, various comparative experiments are performed on synthetic data and mocap data. Compared to other methods, the proposed method is more efficient and accurate