Recovery of Low-Rank Plus Compressed Sparse Matrices with Application to Unveiling Traffic Anomalies

Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability issue arises with traffic anomaly detection in backbone networks, and subsumes compressed sensing as well as the timely low-rank plus sparse matrix recovery tasks encountered in matrix decomposition problems. Leveraging the ability of $\ell_1$- and nuclear norms to recover sparse and low-rank matrices, a convex program is formulated to estimate the unknowns. Analysis and simulations confirm that the said convex program can recover the unknowns for sufficiently low-rank and sparse enough components, along with a compression matrix possessing an isometry property when restricted to operate on sparse vectors. When the low-rank, sparse, and compression matrices are drawn from certain random ensembles, it is established that exact recovery is possible with high probability. First-order algorithms are developed to solve the nonsmooth convex optimization problem with provable iteration complexity guarantees. Insightful tests with synthetic and real network data corroborate the effectiveness of the novel approach in unveiling traffic anomalies across flows and time, and its ability to outperform existing alternatives.Comment: 38 pages, submitted to the IEEE Transactions on Information Theor

Link Delay Estimation via Expander Graphs

One of the purposes of network tomography is to infer the status of parameters (e.g., delay) for the links inside a network through end-to-end probing between (external) boundary nodes along predetermined routes. In this work, we apply concepts from compressed sensing and expander graphs to the delay estimation problem. We first show that a relative majority of network topologies are not expanders for existing expansion criteria. Motivated by this challenge, we then relax such criteria, enabling us to acquire simulation evidence that link delays can be estimated for 30% more networks. That is, our relaxation expands the list of identifiable networks with bounded estimation error by 30%. We conduct a simulation performance analysis of delay estimation and congestion detection on the basis of l1 minimization, demonstrating that accurate estimation is feasible for an increasing proportion of networks