2,887 research outputs found
Structural Analysis of Network Traffic Matrix via Relaxed Principal Component Pursuit
The network traffic matrix is widely used in network operation and
management. It is therefore of crucial importance to analyze the components and
the structure of the network traffic matrix, for which several mathematical
approaches such as Principal Component Analysis (PCA) were proposed. In this
paper, we first argue that PCA performs poorly for analyzing traffic matrix
that is polluted by large volume anomalies, and then propose a new
decomposition model for the network traffic matrix. According to this model, we
carry out the structural analysis by decomposing the network traffic matrix
into three sub-matrices, namely, the deterministic traffic, the anomaly traffic
and the noise traffic matrix, which is similar to the Robust Principal
Component Analysis (RPCA) problem previously studied in [13]. Based on the
Relaxed Principal Component Pursuit (Relaxed PCP) method and the Accelerated
Proximal Gradient (APG) algorithm, we present an iterative approach for
decomposing a traffic matrix, and demonstrate its efficiency and flexibility by
experimental results. Finally, we further discuss several features of the
deterministic and noise traffic. Our study develops a novel method for the
problem of structural analysis of the traffic matrix, which is robust against
pollution of large volume anomalies.Comment: Accepted to Elsevier Computer Network
Recovery of Low-Rank Plus Compressed Sparse Matrices with Application to Unveiling Traffic Anomalies
Given the superposition of a low-rank matrix plus the product of a known fat
compression matrix times a sparse matrix, the goal of this paper is to
establish deterministic conditions under which exact recovery of the low-rank
and sparse components becomes possible. This fundamental identifiability issue
arises with traffic anomaly detection in backbone networks, and subsumes
compressed sensing as well as the timely low-rank plus sparse matrix recovery
tasks encountered in matrix decomposition problems. Leveraging the ability of
- and nuclear norms to recover sparse and low-rank matrices, a convex
program is formulated to estimate the unknowns. Analysis and simulations
confirm that the said convex program can recover the unknowns for sufficiently
low-rank and sparse enough components, along with a compression matrix
possessing an isometry property when restricted to operate on sparse vectors.
When the low-rank, sparse, and compression matrices are drawn from certain
random ensembles, it is established that exact recovery is possible with high
probability. First-order algorithms are developed to solve the nonsmooth convex
optimization problem with provable iteration complexity guarantees. Insightful
tests with synthetic and real network data corroborate the effectiveness of the
novel approach in unveiling traffic anomalies across flows and time, and its
ability to outperform existing alternatives.Comment: 38 pages, submitted to the IEEE Transactions on Information Theor
Decomposable Principal Component Analysis
We consider principal component analysis (PCA) in decomposable Gaussian
graphical models. We exploit the prior information in these models in order to
distribute its computation. For this purpose, we reformulate the problem in the
sparse inverse covariance (concentration) domain and solve the global
eigenvalue problem using a sequence of local eigenvalue problems in each of the
cliques of the decomposable graph. We demonstrate the application of our
methodology in the context of decentralized anomaly detection in the Abilene
backbone network. Based on the topology of the network, we propose an
approximate statistical graphical model and distribute the computation of PCA
Successive Convex Approximation Algorithms for Sparse Signal Estimation with Nonconvex Regularizations
In this paper, we propose a successive convex approximation framework for
sparse optimization where the nonsmooth regularization function in the
objective function is nonconvex and it can be written as the difference of two
convex functions. The proposed framework is based on a nontrivial combination
of the majorization-minimization framework and the successive convex
approximation framework proposed in literature for a convex regularization
function. The proposed framework has several attractive features, namely, i)
flexibility, as different choices of the approximate function lead to different
type of algorithms; ii) fast convergence, as the problem structure can be
better exploited by a proper choice of the approximate function and the
stepsize is calculated by the line search; iii) low complexity, as the
approximate function is convex and the line search scheme is carried out over a
differentiable function; iv) guaranteed convergence to a stationary point. We
demonstrate these features by two example applications in subspace learning,
namely, the network anomaly detection problem and the sparse subspace
clustering problem. Customizing the proposed framework by adopting the
best-response type approximation, we obtain soft-thresholding with exact line
search algorithms for which all elements of the unknown parameter are updated
in parallel according to closed-form expressions. The attractive features of
the proposed algorithms are illustrated numerically.Comment: submitted to IEEE Journal of Selected Topics in Signal Processing,
special issue in Robust Subspace Learnin
Ensemble Joint Sparse Low Rank Matrix Decomposition for Thermography Diagnosis System
Composite is widely used in the aircraft industry and it is essential for manufacturers to monitor its health and quality. The most commonly found defects of composite are debonds and delamination. Different inner defects with complex irregular shape is difficult to be diagnosed by using conventional thermal imaging methods. In this paper, an ensemble joint sparse low rank matrix decomposition (EJSLRMD) algorithm is proposed by applying the optical pulse thermography (OPT) diagnosis system. The proposed algorithm jointly models the low rank and sparse pattern by using concatenated feature space. In particular, the weak defects information can be separated from strong noise and the resolution contrast of the defects has significantly been improved. Ensemble iterative sparse modelling are conducted to further enhance the weak information as well as reducing the computational cost. In order to show the robustness and efficacy of the model, experiments are conducted to detect the inner debond on multiple carbon fiber reinforced polymer (CFRP) composites. A comparative analysis is presented with general OPT algorithms. Not withstand above, the proposed model has been evaluated on synthetic data and compared with other low rank and sparse matrix decomposition algorithms
- …