Towards Informative Statistical Flow Inversion

This is the accepted version of 'Towards Informative Statistical Flow Inversion', archived originally at arXiv:0705.1939v1 [cs.NI] 14 May 2007.A problem which has recently attracted research attention is that of estimating the distribution of flow sizes in internet traffic. On high traffic links it is sometimes impossible to record every packet. Researchers have approached the problem of estimating flow lengths from sampled packet data in two separate ways. Firstly, different sampling methodologies can be tried to more accurately measure the desired system parameters. One such method is the sample-and-hold method where, if a packet is sampled, all subsequent packets in that flow are sampled. Secondly, statistical methods can be used to ``invert'' the sampled data and produce an estimate of flow lengths from a sample. In this paper we propose, implement and test two variants on the sample-and-hold method. In addition we show how the sample-and-hold method can be inverted to get an estimation of the genuine distribution of flow sizes. Experiments are carried out on real network traces to compare standard packet sampling with three variants of sample-and-hold. The methods are compared for their ability to reconstruct the genuine distribution of flow sizes in the traffic

Turbulence in Atomic Hydrogen

Understanding the properties of interstellar turbulence is a great intellectual challenge and the urge to solve this problem is partially motivated by a necessity to explain the star formation mystery. This review deals with a recently suggested inversion technique as applied to atomic hydrogen. This technique allows to determine 3D turbulence statistics through the variations of 21 cm intensity. We claim that a radio interferometer is an ideal tool for such a study as its visibility function is directly related to the statistics of galactic HI. Next, we show how galactic rotation curve can be used to study the turbulence slice by slice and relate the statistics given in galactic coordinates and in the velocity space. The application of the technique to HI data reveals a shallow spectrum of the underlying HI density that is not compatible with a naive Kolmogorov picture. We show that the random density corresponding to the found spectrum tends to form low contrast filaments that are elongated towards the observer.Comment: 9 pages, 2 figures, review, to be published "Interstellar turbulence" CU

Optimal modelling and experimentation for the improved sustainability of microfluidic chemical technology design

Optimization of the dynamics and control of chemical processes holds the promise of improved sustainability for chemical technology by minimizing resource wastage. Anecdotally, chemical plant may be substantially over designed, say by 35-50%, due to designers taking account of uncertainties by providing greater flexibility. Once the plant is commissioned, techniques of nonlinear dynamics analysis can be used by process systems engineers to recoup some of this overdesign by optimization of the plant operation through tighter control. At the design stage, coupling the experimentation with data assimilation into the model, whilst using the partially informed, semi-empirical model to predict from parametric sensitivity studies which experiments to run should optimally improve the model. This approach has been demonstrated for optimal experimentation, but limited to a differential algebraic model of the process. Typically, such models for online monitoring have been limited to low dimensions. Recently it has been demonstrated that inverse methods such as data assimilation can be applied to PDE systems with algebraic constraints, a substantially more complicated parameter estimation using finite element multiphysics modelling. Parametric sensitivity can be used from such semi-empirical models to predict the optimum placement of sensors to be used to collect data that optimally informs the model for a microfluidic sensor system. This coupled optimum modelling and experiment procedure is ambitious in the scale of the modelling problem, as well as in the scale of the application - a microfluidic device. In general, microfluidic devices are sufficiently easy to fabricate, control, and monitor that they form an ideal platform for developing high dimensional spatio-temporal models for simultaneously coupling with experimentation. As chemical microreactors already promise low raw materials wastage through tight control of reagent contacting, improved design techniques should be able to augment optimal control systems to achieve very low resource wastage. In this paper, we discuss how the paradigm for optimal modelling and experimentation should be developed and foreshadow the exploitation of this methodology for the development of chemical microreactors and microfluidic sensors for online monitoring of chemical processes. Improvement in both of these areas bodes to improve the sustainability of chemical processes through innovative technology. (C) 2008 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved

Recombination dynamics of a human Y-chromosomal palindrome:rapid GC-biased gene conversion, multi-kilobase conversion tracts, and rare inversions

The male-specific region of the human Y chromosome (MSY) includes eight large inverted repeats (palindromes) in which arm-to-arm similarity exceeds 99.9%, due to gene conversion activity. Here, we studied one of these palindromes, P6, in order to illuminate the dynamics of the gene conversion process. We genotyped ten paralogous sequence variants (PSVs) within the arms of P6 in 378 Y chromosomes whose evolutionary relationships within the SNP-defined Y phylogeny are known. This allowed the identification of 146 historical gene conversion events involving individual PSVs, occurring at a rate of 2.9-8.4×10(-4) events per generation. A consideration of the nature of nucleotide change and the ancestral state of each PSV showed that the conversion process was significantly biased towards the fixation of G or C nucleotides (GC-biased), and also towards the ancestral state. Determination of haplotypes by long-PCR allowed likely co-conversion of PSVs to be identified, and suggested that conversion tract lengths are large, with a mean of 2068 bp, and a maximum in excess of 9 kb. Despite the frequent formation of recombination intermediates implied by the rapid observed gene conversion activity, resolution via crossover is rare: only three inversions within P6 were detected in the sample. An analysis of chimpanzee and gorilla P6 orthologs showed that the ancestral state bias has existed in all three species, and comparison of human and chimpanzee sequences with the gorilla outgroup confirmed that GC bias of the conversion process has apparently been active in both the human and chimpanzee lineages

Variational data assimilation using targetted random walks

The variational approach to data assimilation is a widely used methodology for both online prediction and for reanalysis (offline hindcasting). In either of these scenarios it can be important to assess uncertainties in the assimilated state. Ideally it would be desirable to have complete information concerning the Bayesian posterior distribution for unknown state, given data. The purpose of this paper is to show that complete computational probing of this posterior distribution is now within reach in the offline situation. In this paper we will introduce an MCMC method which enables us to directly sample from the Bayesian\ud posterior distribution on the unknown functions of interest, given observations. Since we are aware that these\ud methods are currently too computationally expensive to consider using in an online filtering scenario, we frame this in the context of offline reanalysis. Using a simple random walk-type MCMC method, we are able to characterize the posterior distribution using only evaluations of the forward model of the problem, and of the model and data mismatch. No adjoint model is required for the method we use; however more sophisticated MCMC methods are available\ud which do exploit derivative information. For simplicity of exposition we consider the problem of assimilating data, either Eulerian or Lagrangian, into a low Reynolds number (Stokes flow) scenario in a two dimensional periodic geometry. We will show that in many cases it is possible to recover the initial condition and model error (which we describe as unknown forcing to the model) from data, and that with increasing amounts of informative data, the uncertainty in our estimations reduces

Bayesian power-spectrum inference for Large Scale Structure data

We describe an exact, flexible, and computationally efficient algorithm for a joint estimation of the large-scale structure and its power-spectrum, building on a Gibbs sampling framework and present its implementation ARES (Algorithm for REconstruction and Sampling). ARES is designed to reconstruct the 3D power-spectrum together with the underlying dark matter density field in a Bayesian framework, under the reasonable assumption that the long wavelength Fourier components are Gaussian distributed. As a result ARES does not only provide a single estimate but samples from the joint posterior of the power-spectrum and density field conditional on a set of observations. This enables us to calculate any desired statistical summary, in particular we are able to provide joint uncertainty estimates. We apply our method to mock catalogs, with highly structured observational masks and selection functions, in order to demonstrate its ability to reconstruct the power-spectrum from real data sets, while fully accounting for any mask induced mode coupling.Comment: 25 pages, 15 figure