Structural Analysis of Network Traffic Matrix via Relaxed Principal Component Pursuit

The network traffic matrix is widely used in network operation and management. It is therefore of crucial importance to analyze the components and the structure of the network traffic matrix, for which several mathematical approaches such as Principal Component Analysis (PCA) were proposed. In this paper, we first argue that PCA performs poorly for analyzing traffic matrix that is polluted by large volume anomalies, and then propose a new decomposition model for the network traffic matrix. According to this model, we carry out the structural analysis by decomposing the network traffic matrix into three sub-matrices, namely, the deterministic traffic, the anomaly traffic and the noise traffic matrix, which is similar to the Robust Principal Component Analysis (RPCA) problem previously studied in [13]. Based on the Relaxed Principal Component Pursuit (Relaxed PCP) method and the Accelerated Proximal Gradient (APG) algorithm, we present an iterative approach for decomposing a traffic matrix, and demonstrate its efficiency and flexibility by experimental results. Finally, we further discuss several features of the deterministic and noise traffic. Our study develops a novel method for the problem of structural analysis of the traffic matrix, which is robust against pollution of large volume anomalies.Comment: Accepted to Elsevier Computer Network

Network traffic anomaly detection using EMD and Hilbert-Huan transform

Empirical Mode Decomposition (EMD) and Hilbert-Huang Transform (HHT) provide a means for adaptive data analysis. EMD extracts Intrinsic Mode Functions (IMFs) that represent the frequency and amplitude characteristics of a signal. HHT generates the marginal spectrum and energy density level of a signal. The IMFs, the marginal spectrum, and the energy density level characterize a signal from three different perspectives. This thesis proposes three novel parameters for network traffic anomaly detection based on the above three signal characteristics. Hurst parameter of network traffic is calculated based on the first IMF, and is expanded by introducing a weighted self-similarity based on the concept of entropy. Pearson’s distance is calculated based on the marginal spectrum to differentiate normal traffic from abnormal ones. Finally, the slopes of crosscorrelations are calculated based on the energy density level to detect the rate of energy change between normal and abnormal internet traffic