Tensor Completion via Leverage Sampling and Tensor QR Decomposition for Network Latency Estimation

In this paper, we consider the network latency estimation, which has been an important metric for network performance. However, a large scale of network latency estimation requires a lot of computing time. Therefore, we propose a new method that is much faster and maintains high accuracy. The data structure of network nodes can form a matrix, and the tensor model can be formed by introducing the time dimension. Thus, the entire problem can be be summarized as a tensor completion problem. The main idea of our method is improving the tensor leverage sampling strategy and introduce tensor QR decomposition into tensor completion. To achieve faster tensor leverage sampling, we replace tensor singular decomposition (t-SVD) with tensor CSVD-QR to appoximate t-SVD. To achieve faster completion for incomplete tensor, we use the tensor $L_{2,1}$-norm rather than traditional tensor nuclear norm. Furthermore, we introduce tensor QR decomposition into alternating direction method of multipliers (ADMM) framework. Numerical experiments witness that our method is faster than state-of-art algorithms with satisfactory accuracy.Comment: 20 pages, 7 figure

Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time

Given a matrix $M\in \mathbb{R}^{m\times n}$, the low rank matrix completion problem asks us to find a rank-$k$ approximation of $M$ as $UV^\top$ for $U\in \mathbb{R}^{m\times k}$ and $V\in \mathbb{R}^{n\times k}$ by only observing a few entries specified by a set of entries $\Omega\subseteq [m]\times [n]$. In particular, we examine an approach that is widely used in practice -- the alternating minimization framework. Jain, Netrapalli and Sanghavi~\cite{jns13} showed that if $M$ has incoherent rows and columns, then alternating minimization provably recovers the matrix $M$ by observing a nearly linear in $n$ number of entries. While the sample complexity has been subsequently improved~\cite{glz17}, alternating minimization steps are required to be computed exactly. This hinders the development of more efficient algorithms and fails to depict the practical implementation of alternating minimization, where the updates are usually performed approximately in favor of efficiency. In this paper, we take a major step towards a more efficient and error-robust alternating minimization framework. To this end, we develop an analytical framework for alternating minimization that can tolerate moderate amount of errors caused by approximate updates. Moreover, our algorithm runs in time $\widetilde O(|\Omega| k)$, which is nearly linear in the time to verify the solution while preserving the sample complexity. This improves upon all prior known alternating minimization approaches which require $\widetilde O(|\Omega| k^2)$ time.Comment: Improve the runtime from $O(mnk)$ to $O|\Omega| k)

Traffic matrix estimation with enhanced origin destination generator algorithm using simulation of real network

The rapid growth of the Internet has made the issue of ensuring reliability and redundancy a big challenge. Studies of these issues using Traffic Engineering and simulation have been extensively done. In Traffic Matrix Estimation (TME), the Origin–Destination Generator algorithm (ODGen) is limited to the number of hops, where the Expectation Maximization (EM) accuracy is 92%. Most studies have not taken into account real traffic parameters and integration of TME models with routing protocols in their simulation models. Also, there is no a comprehensive model consisting of TME, Border Gateway Protocol (BGP) and Hot Potato (HP) routing in the NS-2 network simulator based on real networks. In this research, Integrated Simulated Model (ISM) is introduced consisting of ODGen-HP algorithm and BGP integrated into the NS-2 network simulator. ISM is then used to simulate the infrastructure of a real production network using actual captured traffic data parameters. Validation is then done against the changes in network topology based on packet loss, delay and throughput. Results gave the average error for packet sent by simulated and production networks of 0% and the average error for packet received by simulation and production networks of 3.61%. The network is modelled with a baseline topology where 5 main nodes were connected together, with redundant links for some nodes. The simulations were repeated for link failures, node addition, and node removal. TME used in ISM is based on ODGen, that is optimized with unlimited number of hops, the accuracy of EM increases to 97% and Central Processing Unit complexity is reduced. HP helps in improving the node which experiences a link failure to select shorter distance route to egress router. In the case of a link failure, HP switching time between the links is 0.05 seconds. ISM performance was evaluated by comparing trace file before and after link failure or by adding nodes (up to 32) or removing nodes. The parameters used for comparison are the packets loss, delay and throughput. The ISM error percentage obtained for packets loss is 0.025%, delay 0.013% and throughput 0.003%

On Traffic Matrix Completion in the Internet

The ability of an ISP to infer traffic volumes that are not directly measurable can be useful for research, engineering, and business intelligence. Previous work has shown that traffic matrix completion is possible, but there is as yet no clear understanding of which ASes are likely to be able to perform TM completion, and which traffic flows can be inferred. In this paper we investigate the relationship between the ASlevel topology of the Internet and the ability of an individual AS to perform traffic matrix completion. We take a three-stage approach, starting from abstract analysis on idealized topologies, and then adding realistic routing and topologies, and finally incorporating realistic traffic on which we perform actual TM completion. Our first set of results identifies which ASes are bestpositioned to perform TM completion. We show, surprisingly, that for TM completion it does not help for an AS to have many peering links. Rather, the most important factor enabling an AS to perform TM completion is the number of direct customers it has. Our second set of results focuses on which flows can be inferred. We show that topologically close flows are easier to infer, and that flows passing through customers are particularly well suited for inference