540 research outputs found
A Two-step Statistical Approach for Inferring Network Traffic Demands (Revises Technical Report BUCS-2003-003)
Accurate knowledge of traffic demands in a communication network enables or enhances a variety of traffic engineering and network management tasks of paramount importance for operational networks. Directly measuring a complete set of these demands is prohibitively expensive because of the huge amounts of data that must be collected and the performance impact that such measurements would impose on the regular behavior of the network. As a consequence, we must rely on statistical techniques to produce estimates of actual traffic demands from partial information. The performance of such techniques is however limited due to their reliance on limited information and the high amount of computations they incur, which limits their convergence behavior. In this paper we study a two-step approach for inferring network traffic demands. First we elaborate and evaluate a modeling approach for generating good starting points to be fed to iterative statistical inference techniques. We call these starting points informed priors since they are obtained using actual network information such as packet traces and SNMP link counts. Second we provide a very fast variant of the EM algorithm which extends its computation range, increasing its accuracy and decreasing its dependence on the quality of the starting point. Finally, we evaluate and compare alternative mechanisms for generating starting points and the convergence characteristics of our EM algorithm against a recently proposed Weighted Least Squares approach.National Science Foundation (ANI-0095988, EIA-0202067, ITR ANI-0205294
Information Recovery In Behavioral Networks
In the context of agent based modeling and network theory, we focus on the
problem of recovering behavior-related choice information from
origin-destination type data, a topic also known under the name of network
tomography. As a basis for predicting agents' choices we emphasize the
connection between adaptive intelligent behavior, causal entropy maximization
and self-organized behavior in an open dynamic system. We cast this problem in
the form of binary and weighted networks and suggest information theoretic
entropy-driven methods to recover estimates of the unknown behavioral flow
parameters. Our objective is to recover the unknown behavioral values across
the ensemble analytically, without explicitly sampling the configuration space.
In order to do so, we consider the Cressie-Read family of entropic functionals,
enlarging the set of estimators commonly employed to make optimal use of the
available information. More specifically, we explicitly work out two cases of
particular interest: Shannon functional and the likelihood functional. We then
employ them for the analysis of both univariate and bivariate data sets,
comparing their accuracy in reproducing the observed trends.Comment: 14 pages, 6 figures, 4 table
Network Tomography: Identifiability and Fourier Domain Estimation
The statistical problem for network tomography is to infer the distribution
of , with mutually independent components, from a measurement model
, where is a given binary matrix representing the
routing topology of a network under consideration. The challenge is that the
dimension of is much larger than that of and thus the
problem is often called ill-posed. This paper studies some statistical aspects
of network tomography. We first address the identifiability issue and prove
that the distribution is identifiable up to a shift parameter
under mild conditions. We then use a mixture model of characteristic functions
to derive a fast algorithm for estimating the distribution of
based on the General method of Moments. Through extensive model simulation and
real Internet trace driven simulation, the proposed approach is shown to be
favorable comparing to previous methods using simple discretization for
inferring link delays in a heterogeneous network.Comment: 21 page
Estimating point-to-point and point-to-multipoint traffic matrices: An information-theoretic approach
© 2005 IEEE.Traffic matrices are required inputs for many IP network management tasks, such as capacity planning, traffic engineering, and network reliability analysis. However, it is difficult to measure these matrices directly in large operational IP networks, so there has been recent interest in inferring traffic matrices from link measurements and other more easily measured data. Typically, this inference problem is ill-posed, as it involves significantly more unknowns than data. Experience in many scientific and engineering fields has shown that it is essential to approach such ill-posed problems via "regularization". This paper presents a new approach to traffic matrix estimation using a regularization based on "entropy penalization". Our solution chooses the traffic matrix consistent with the measured data that is information-theoretically closest to a model in which source/destination pairs are stochastically independent. It applies to both point-to-point and point-to-multipoint traffic matrix estimation. We use fast algorithms based on modern convex optimization theory to solve for our traffic matrices. We evaluate our algorithm with real backbone traffic and routing data, and demonstrate that it is fast, accurate, robust, and flexible.Yin Zhang, Member, Matthew Roughan, Carsten Lund, and David L. Donoh
Bayesian Inference for Static Traffic Network Flows with Mobile Sensor Data
Vehicle trajectory information are becoming available from mobile sensors such as onboard devices or smart phones. Such data can provide partial information of origin-destination trips and are very helpful in solving the network flow estimation problem which can be very challenging if only link counts are used. Even with this new information, however, there is still structural bias in the maximum likelihood based approach because of uncertainties in the penetration rates. A Bayesian inference approach in which the earlier link-count-based methods are extended is proposed. We incorporate posterior simulation of route-choice probabilities and penetration rates. The results of a numerical example show that our method can infer network flow parameters effectively. Inclusion of mobile sensor data and prior beliefs based on it can yield much better inference results than when non-informative priors and only link counts are used
- …