19,022 research outputs found
Network Kriging
Network service providers and customers are often concerned with aggregate
performance measures that span multiple network paths. Unfortunately, forming
such network-wide measures can be difficult, due to the issues of scale
involved. In particular, the number of paths grows too rapidly with the number
of endpoints to make exhaustive measurement practical. As a result, it is of
interest to explore the feasibility of methods that dramatically reduce the
number of paths measured in such situations while maintaining acceptable
accuracy.
We cast the problem as one of statistical prediction--in the spirit of the
so-called `kriging' problem in spatial statistics--and show that end-to-end
network properties may be accurately predicted in many cases using a
surprisingly small set of carefully chosen paths. More precisely, we formulate
a general framework for the prediction problem, propose a class of linear
predictors for standard quantities of interest (e.g., averages, totals,
differences) and show that linear algebraic methods of subset selection may be
used to effectively choose which paths to measure. We characterize the
performance of the resulting methods, both analytically and numerically. The
success of our methods derives from the low effective rank of routing matrices
as encountered in practice, which appears to be a new observation in its own
right with potentially broad implications on network measurement generally.Comment: 16 pages, 9 figures, single-space
Empirical stationary correlations for semi-supervised learning on graphs
In semi-supervised learning on graphs, response variables observed at one
node are used to estimate missing values at other nodes. The methods exploit
correlations between nearby nodes in the graph. In this paper we prove that
many such proposals are equivalent to kriging predictors based on a fixed
covariance matrix driven by the link structure of the graph. We then propose a
data-driven estimator of the correlation structure that exploits patterns among
the observed response values. By incorporating even a small fraction of
observed covariation into the predictions, we are able to obtain much improved
prediction on two graph data sets.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS293 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Leak localization in water distribution networks using pressure and data-driven classifier approach
Leaks in water distribution networks (WDNs) are one of the main reasons for water loss during fluid transportation. Considering the worldwide problem of water scarcity, added to the challenges that a growing population brings, minimizing water losses through leak detection and localization, timely and efficiently using advanced techniques is an urgent humanitarian need. There are numerous methods being used to localize water leaks in WDNs through constructing hydraulic models or analyzing flow/pressure deviations between the observed data and the estimated values. However, from the application perspective, it is very practical to implement an approach which does not rely too much on measurements and complex models with reasonable computation demand. Under this context, this paper presents a novel method for leak localization which uses a data-driven approach based on limit pressure measurements in WDNs with two stages included: (1) Two different machine learning classifiers based on linear discriminant analysis (LDA) and neural networks (NNET) are developed to determine the probabilities of each node having a leak inside a WDN; (2) Bayesian temporal reasoning is applied afterwards to rescale the probabilities of each possible leak location at each time step after a leak is detected, with the aim of improving the localization accuracy. As an initial illustration, the hypothetical benchmark Hanoi district metered area (DMA) is used as the case study to test the performance of the proposed approach. Using the fitting accuracy and average topological distance (ATD) as performance indicators, the preliminary results reaches more than 80% accuracy in the best cases.Peer ReviewedPostprint (published version
Global Modeling and Prediction of Computer Network Traffic
We develop a probabilistic framework for global modeling of the traffic over
a computer network. This model integrates existing single-link (-flow) traffic
models with the routing over the network to capture the global traffic
behavior. It arises from a limit approximation of the traffic fluctuations as
the time--scale and the number of users sharing the network grow. The resulting
probability model is comprised of a Gaussian and/or a stable, infinite variance
components. They can be succinctly described and handled by certain
'space-time' random fields. The model is validated against simulated and real
data. It is then applied to predict traffic fluctuations over unobserved links
from a limited set of observed links. Further, applications to anomaly
detection and network management are briefly discussed
- …