Inverting sampled ADSL traffic

On the basis of a reference model for ADSL traffic on an IP backbone link, established in an earlier study, we show in this paper that it is possible to infer the characteristics of long flows by performing a deterministic $1/N$ packet sampling. By using the fact that the number of active long flows can be represented by means of the number of customers in an $M/G/\infty$ queue with Weibullian service times, we derive some probabilistic properties of sampled data. These properties are then used to infer the characteristics of original flows. The method is illustrated by considering an actual traffic trace captured in the France Telecom IP backbone network. Experimental data show that the method proves quite efficient

Towards Informative Statistical Flow Inversion

This is the accepted version of 'Towards Informative Statistical Flow Inversion', archived originally at arXiv:0705.1939v1 [cs.NI] 14 May 2007.A problem which has recently attracted research attention is that of estimating the distribution of flow sizes in internet traffic. On high traffic links it is sometimes impossible to record every packet. Researchers have approached the problem of estimating flow lengths from sampled packet data in two separate ways. Firstly, different sampling methodologies can be tried to more accurately measure the desired system parameters. One such method is the sample-and-hold method where, if a packet is sampled, all subsequent packets in that flow are sampled. Secondly, statistical methods can be used to ``invert'' the sampled data and produce an estimate of flow lengths from a sample. In this paper we propose, implement and test two variants on the sample-and-hold method. In addition we show how the sample-and-hold method can be inverted to get an estimation of the genuine distribution of flow sizes. Experiments are carried out on real network traces to compare standard packet sampling with three variants of sample-and-hold. The methods are compared for their ability to reconstruct the genuine distribution of flow sizes in the traffic

Non-inverting Tillage: Early-Stage Effects on Soil Mechanical Behaviour

Organic farmers often claim positive effects of non-inverting and reduced tillage systems. There is a need of quantifying tilth characteristics in the former plough layer of soil converted to such tillage systems. A non-inverting tillage system (NINV) was tested in a field experiment conducted on a Danish sandy loam soil. It included deep loosening and shallow intensive cultivation and was compared to a conventional ploughing-harrowing tillage system (CONV). A hierarchical analytical procedure was applied in studies of soil fragmentation and soil strength characteristics for the 7-14 cm soil layer. A visual description was carried out and ease of fragmentation was evaluated in the field using a soil drop test. Soil strength was measured in the field with a cone penetrometer and a torsional shear box method, and in the laboratory using an annulus shear strength method. Tensile strength was determined in the laboratory on field-sampled aggregates. The CONV treated soil displayed a higher ease of fragmentation in the field in May as well as in September. In general, aggregates from the NINV treated soil were stronger than aggregates from the CONV treatment. The soils had similar friability indices in May. In September, however, a higher friability index was found for the CONV treated soil (k=0.22 and 0.16, respectively for CONV and NINV). The NINV treated soil also displayed the highest soil strength. The soil tilth was evaluated to be best in the CONV treated soil. Supposed meliorating actions during the growing season did not eliminate the differences between the treatments

Lagrangian-based Hydrodynamic Model: Freeway Traffic Estimation

This paper is concerned with highway traffic estimation using traffic sensing data, in a Lagrangian-based modeling framework. We consider the Lighthill-Whitham-Richards (LWR) model (Lighthill and Whitham, 1955; Richards, 1956) in Lagrangian-coordinates, and provide rigorous mathematical results regarding the equivalence of viscosity solutions to the Hamilton-Jacobi equations in Eulerian and Lagrangian coordinates. We derive closed-form solutions to the Lagrangian-based Hamilton-Jacobi equation using the Lax-Hopf formula (Daganzo, 2005; Aubin et al., 2008), and discuss issues of fusing traffic data of various types into the Lagrangian-based H-J equation. A numerical study of the Mobile Century field experiment (Herrera et al., 2009) demonstrates the unique modeling features and insights provided by the Lagrangian-based approach.Comment: 17 pages, 7 figures, current version submitted to Transportation Research Part

Stream Sampling for Frequency Cap Statistics

Unaggregated data, in streamed or distributed form, is prevalent and come from diverse application domains which include interactions of users with web services and IP traffic. Data elements have {\em keys} (cookies, users, queries) and elements with different keys interleave. Analytics on such data typically utilizes statistics stated in terms of the frequencies of keys. The two most common statistics are {\em distinct}, which is the number of active keys in a specified segment, and {\em sum}, which is the sum of the frequencies of keys in the segment. Both are special cases of {\em cap} statistics, defined as the sum of frequencies {\em capped} by a parameter $T$, which are popular in online advertising platforms. Aggregation by key, however, is costly, requiring state proportional to the number of distinct keys, and therefore we are interested in estimating these statistics or more generally, sampling the data, without aggregation. We present a sampling framework for unaggregated data that uses a single pass (for streams) or two passes (for distributed data) and state proportional to the desired sample size. Our design provides the first effective solution for general frequency cap statistics. Our $\ell$-capped samples provide estimates with tight statistical guarantees for cap statistics with $T=\Theta(\ell)$ and nonnegative unbiased estimates of {\em any} monotone non-decreasing frequency statistics. An added benefit of our unified design is facilitating {\em multi-objective samples}, which provide estimates with statistical guarantees for a specified set of different statistics, using a single, smaller sample.Comment: 21 pages, 4 figures, preliminary version will appear in KDD 201

Advanced Multi-Channel SAR Imaging - Measured Data Demonstration

Synthetic Aperture Radar (SAR) is a well-established technique for remote sensing of the Earth. However, conventional SAR systems relying on only a single transmit and receive aperture are not capable of imaging a wide swath with high spatial resolution. Multi-channel SAR concepts, such as systems based on multiple receive apertures in azimuth, promise to overcome these restrictions, thus enabling high-resolution wide-swath imaging. Analysis revealed that these systems imperatively require sophisticated digital processing of the received signals in order to guarantee full performance independently of the spatial sample distribution imposed by the applied pulse repetition frequency (PRF). A suitable algorithm to cope with these challenges of multi-channel data is given by the “multi-channel reconstruction algorithm”, which demonstrated in comprehensive analysis and system design examples its potential for high perform-ance SAR imaging. In this context, various optimization strategies were investigated and aspects of operating multi-channel systems in burst modes such as ScanSAR or TOPS were discussed. Furthermore, a first proof-of-principle showed the algorithm’s applicability to measured multi-channel X-band data gathered by the German Aerospace Cen-ter’s (DLR) airborne F-SAR system. As a next step in the framework of multi-channel azimuth processing, this paper builds on the results recalled above and continues two paths. Firstly, focus is turned to further optimization of the proc-essing algorithm by investigating the classical Space-Time Adaptive Processing (STAP) applied to SAR. Secondly, attention is turned to the analysis of the measured multi-channel data by elaborating the impact and compensation of channel mismatch and by verifying the derived theory