3,297 research outputs found
An Improved Traffic Matrix Decomposition Method with Frequency-Domain Regularization
We propose a novel network traffic matrix decomposition method named Stable
Principal Component Pursuit with Frequency-Domain Regularization (SPCP-FDR),
which improves the Stable Principal Component Pursuit (SPCP) method by using a
frequency-domain noise regularization function. An experiment demonstrates the
feasibility of this new decomposition method.Comment: Accepted to IEICE Transactions on Information and System
Scalable Tensor Factorizations for Incomplete Data
The problem of incomplete data - i.e., data with missing or unknown values -
in multi-way arrays is ubiquitous in biomedical signal processing, network
traffic analysis, bibliometrics, social network analysis, chemometrics,
computer vision, communication networks, etc. We consider the problem of how to
factorize data sets with missing values with the goal of capturing the
underlying latent structure of the data and possibly reconstructing missing
values (i.e., tensor completion). We focus on one of the most well-known tensor
factorizations that captures multi-linear structure, CANDECOMP/PARAFAC (CP). In
the presence of missing data, CP can be formulated as a weighted least squares
problem that models only the known entries. We develop an algorithm called
CP-WOPT (CP Weighted OPTimization) that uses a first-order optimization
approach to solve the weighted least squares problem. Based on extensive
numerical experiments, our algorithm is shown to successfully factorize tensors
with noise and up to 99% missing data. A unique aspect of our approach is that
it scales to sparse large-scale data, e.g., 1000 x 1000 x 1000 with five
million known entries (0.5% dense). We further demonstrate the usefulness of
CP-WOPT on two real-world applications: a novel EEG (electroencephalogram)
application where missing data is frequently encountered due to disconnections
of electrodes and the problem of modeling computer network traffic where data
may be absent due to the expense of the data collection process
Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach
Traffic speed is central to characterizing the fluidity of the road network.
Many transportation applications rely on it, such as real-time navigation,
dynamic route planning, and congestion management. Rapid advances in sensing
and communication techniques make traffic speed detection easier than ever.
However, due to sparse deployment of static sensors or low penetration of
mobile sensors, speeds detected are incomplete and far from network-wide use.
In addition, sensors are prone to error or missing data due to various kinds of
reasons, speeds from these sensors can become highly noisy. These drawbacks
call for effective techniques to recover credible estimates from the incomplete
data. In this work, we first identify the issue as a spatiotemporal kriging
problem and propose a Laplacian enhanced low-rank tensor completion (LETC)
framework featuring both lowrankness and multi-dimensional correlations for
large-scale traffic speed kriging under limited observations. To be specific,
three types of speed correlation including temporal continuity, temporal
periodicity, and spatial proximity are carefully chosen and simultaneously
modeled by three different forms of graph Laplacian, named temporal graph
Fourier transform, generalized temporal consistency regularization, and
diffusion graph regularization. We then design an efficient solution algorithm
via several effective numeric techniques to scale up the proposed model to
network-wide kriging. By performing experiments on two public million-level
traffic speed datasets, we finally draw the conclusion and find our proposed
LETC achieves the state-of-the-art kriging performance even under low
observation rates, while at the same time saving more than half computing time
compared with baseline methods. Some insights into spatiotemporal traffic data
modeling and kriging at the network level are provided as well
Spatiotemporal Tensor Completion for Improved Urban Traffic Imputation
Effective management of urban traffic is important for any smart city
initiative. Therefore, the quality of the sensory traffic data is of paramount
importance. However, like any sensory data, urban traffic data are prone to
imperfections leading to missing measurements. In this paper, we focus on
inter-region traffic data completion. We model the inter-region traffic as a
spatiotemporal tensor that suffers from missing measurements. To recover the
missing data, we propose an enhanced CANDECOMP/PARAFAC (CP) completion approach
that considers the urban and temporal aspects of the traffic. To derive the
urban characteristics, we divide the area of study into regions. Then, for each
region, we compute urban feature vectors inspired from biodiversity which are
used to compute the urban similarity matrix. To mine the temporal aspect, we
first conduct an entropy analysis to determine the most regular time-series.
Then, we conduct a joint Fourier and correlation analysis to compute its
periodicity and construct the temporal matrix. Both urban and temporal matrices
are fed into a modified CP-completion objective function. To solve this
objective, we propose an alternating least square approach that operates on the
vectorized version of the inputs. We conduct comprehensive comparative study
with two evaluation scenarios. In the first one, we simulate random missing
values. In the second scenario, we simulate missing values at a given area and
time duration. Our results demonstrate that our approach provides effective
recovering performance reaching 26% improvement compared to state-of-art CP
approaches and 35% compared to state-of-art generative model-based approaches
- …