Modeling Massive Amount of Experimental Data with Large Random Matrices in a Real-Time UWB-MIMO System

The aim of this paper is to study data modeling for massive datasets. Large random matrices are used to model the massive amount of data collected from our experimental testbed. This testbed was developed for a real-time ultra-wideband, multiple input multiple output (UWB-MIMO) system. Empirical spectral density is the relevant information we seek for. After we treat this UWB-MIMO system as a black box, we aim to model the output of the black box as a large statistical system, whose outputs can be described by (large) random matrices. This model is extremely general to allow for the study of non-linear and non-Gaussian phenomenon. The good agreements between the theoretical predictions and the empirical findings validate the correctness of the our suggested data model.Comment: 4 pages, 11 figure

3D Power-map for Smart Grids---An Integration of High-dimensional Analysis and Visualization

Data with features of volume, velocity, variety, and veracity are challenging traditional tools to extract useful analysis for decision-making. By integrating high-dimensional analysis with visualization, this paper develops a 3D power-map animation as an effective solution to the challenge. An architecture design, with detailed data processing procedure, is proposed to realize the integration. Two of the most important components in the architecture are presented: the Single-Ring Law for random matrices as solid mathematic foundation, and the proposed statistical index MSR as high-dimensional data for visualization. The whole procedure is easy in logic, fast in speed, objective and even robust against bad data. Moreover, it is an unsupervised machine learning mechanism directly oriented to the raw data rather than logics or models based on simplifications and assumptions. A case study validates the effectiveness and performance of the developed 3D power-map in analysis extraction.Comment: 5 pages, 7 figures, submitted to PESGM 2015. arXiv admin note: substantial text overlap with arXiv:1502.0006

A Random Matrix Theoretical Approach to Early Event Detection in Smart Grid

Power systems are developing very fast nowadays, both in size and in complexity; this situation is a challenge for Early Event Detection (EED). This paper proposes a data- driven unsupervised learning method to handle this challenge. Specifically, the random matrix theories (RMTs) are introduced as the statistical foundations for random matrix models (RMMs); based on the RMMs, linear eigenvalue statistics (LESs) are defined via the test functions as the system indicators. By comparing the values of the LES between the experimental and the theoretical ones, the anomaly detection is conducted. Furthermore, we develop 3D power-map to visualize the LES; it provides a robust auxiliary decision-making mechanism to the operators. In this sense, the proposed method conducts EED with a pure statistical procedure, requiring no knowledge of system topologies, unit operation/control models, etc. The LES, as a key ingredient during this procedure, is a high dimensional indictor derived directly from raw data. As an unsupervised learning indicator, the LES is much more sensitive than the low dimensional indictors obtained from supervised learning. With the statistical procedure, the proposed method is universal and fast; moreover, it is robust against traditional EED challenges (such as error accumulations, spurious correlations, and even bad data in core area). Case studies, with both simulated data and real ones, validate the proposed method. To manage large-scale distributed systems, data fusion is mentioned as another data processing ingredient.Comment: 12 pages, 11 figures, submitted to IEEE Transactions on Smart Gri

Spatio-Temporal Big Data Analysis for Smart Grids Based on Random Matrix Theory: A Comprehensive Study

A cornerstone of the smart grid is the advanced monitorability on its assets and operations. Increasingly pervasive installation of the phasor measurement units (PMUs) allows the so-called synchrophasor measurements to be taken roughly 100 times faster than the legacy supervisory control and data acquisition (SCADA) measurements, time-stamped using the global positioning system (GPS) signals to capture the grid dynamics. On the other hand, the availability of low-latency two-way communication networks will pave the way to high-precision real-time grid state estimation and detection, remedial actions upon network instability, and accurate risk analysis and post-event assessment for failure prevention. In this chapter, we firstly modelling spatio-temporal PMU data in large scale grids as random matrix sequences. Secondly, some basic principles of random matrix theory (RMT), such as asymptotic spectrum laws, transforms, convergence rate and free probability, are introduced briefly in order to the better understanding and application of RMT technologies. Lastly, the case studies based on synthetic data and real data are developed to evaluate the performance of the RMT-based schemes in different application scenarios (i.e., state evaluation and situation awareness).Comment: Book chapter#23 for the book "Transportation and Power Grid in Smart Cities: Communication Networks and Services". arXiv admin note: text overlap with arXiv:1302.0885 by other author