Pointwise Relations between Information and Estimation in Gaussian Noise

Many of the classical and recent relations between information and estimation in the presence of Gaussian noise can be viewed as identities between expectations of random quantities. These include the I-MMSE relationship of Guo et al.; the relative entropy and mismatched estimation relationship of Verd\'{u}; the relationship between causal estimation and mutual information of Duncan, and its extension to the presence of feedback by Kadota et al.; the relationship between causal and non-casual estimation of Guo et al., and its mismatched version of Weissman. We dispense with the expectations and explore the nature of the pointwise relations between the respective random quantities. The pointwise relations that we find are as succinctly stated as - and give considerable insight into - the original expectation identities. As an illustration of our results, consider Duncan's 1970 discovery that the mutual information is equal to the causal MMSE in the AWGN channel, which can equivalently be expressed saying that the difference between the input-output information density and half the causal estimation error is a zero mean random variable (regardless of the distribution of the channel input). We characterize this random variable explicitly, rather than merely its expectation. Classical estimation and information theoretic quantities emerge with new and surprising roles. For example, the variance of this random variable turns out to be given by the causal MMSE (which, in turn, is equal to the mutual information by Duncan's result).Comment: 31 pages, 2 figures, submitted to IEEE Transactions on Information Theor

Hydrological controls on river network connectivity

This study proposes a probabilistic approach for the quantitative assessment of reach- and network-scale hydrological connectivity as dictated by river flow space–time variability. Spatial dynamics of daily streamflows are estimated based on climatic and morphological features of the contributing catchment, integrating a physically based approach that accounts for the stochasticity of rainfall with a water balance framework and a geomorphic recession flow analysis. Ecologically meaningful minimum stage thresholds are used to evaluate the connectivity of individual stream reaches, and other relevant network-scale connectivity metrics. The framework allows a quantitative description of the main hydrological causes and the ecological consequences of water depth dynamics experienced by river networks. The analysis shows that the spatial variability of local-scale hydrological connectivity is strongly affected by the spatial and temporal distribution of climatic variables. Depending on the underlying climatic settings and the critical stage threshold, loss of connectivity can be observed in the headwaters or along the main channel, thereby originating a fragmented river network. The proposed approach provides important clues for understanding the effect of climate on the ecological function of river corridors

Optimal modelling and experimentation for the improved sustainability of microfluidic chemical technology design

Optimization of the dynamics and control of chemical processes holds the promise of improved sustainability for chemical technology by minimizing resource wastage. Anecdotally, chemical plant may be substantially over designed, say by 35-50%, due to designers taking account of uncertainties by providing greater flexibility. Once the plant is commissioned, techniques of nonlinear dynamics analysis can be used by process systems engineers to recoup some of this overdesign by optimization of the plant operation through tighter control. At the design stage, coupling the experimentation with data assimilation into the model, whilst using the partially informed, semi-empirical model to predict from parametric sensitivity studies which experiments to run should optimally improve the model. This approach has been demonstrated for optimal experimentation, but limited to a differential algebraic model of the process. Typically, such models for online monitoring have been limited to low dimensions. Recently it has been demonstrated that inverse methods such as data assimilation can be applied to PDE systems with algebraic constraints, a substantially more complicated parameter estimation using finite element multiphysics modelling. Parametric sensitivity can be used from such semi-empirical models to predict the optimum placement of sensors to be used to collect data that optimally informs the model for a microfluidic sensor system. This coupled optimum modelling and experiment procedure is ambitious in the scale of the modelling problem, as well as in the scale of the application - a microfluidic device. In general, microfluidic devices are sufficiently easy to fabricate, control, and monitor that they form an ideal platform for developing high dimensional spatio-temporal models for simultaneously coupling with experimentation. As chemical microreactors already promise low raw materials wastage through tight control of reagent contacting, improved design techniques should be able to augment optimal control systems to achieve very low resource wastage. In this paper, we discuss how the paradigm for optimal modelling and experimentation should be developed and foreshadow the exploitation of this methodology for the development of chemical microreactors and microfluidic sensors for online monitoring of chemical processes. Improvement in both of these areas bodes to improve the sustainability of chemical processes through innovative technology. (C) 2008 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved

Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network

With the growing popularity of smartphones, capturing high-quality images is of vital importance to smartphones. The cameras of smartphones have small apertures and small sensor cells, which lead to the noisy images in low light environment. Denoising based on a burst of multiple frames generally outperforms single frame denoising but with the larger compututional cost. In this paper, we propose an efficient yet effective burst denoising system. We adopt a three-stage design: noise prior integration, multi-frame alignment and multi-frame denoising. First, we integrate noise prior by pre-processing raw signals into a variance-stabilization space, which allows using a small-scale network to achieve competitive performance. Second, we observe that it is essential to adopt an explicit alignment for burst denoising, but it is not necessary to integrate a learning-based method to perform multi-frame alignment. Instead, we resort to a conventional and efficient alignment method and combine it with our multi-frame denoising network. At last, we propose a denoising strategy that processes multiple frames sequentially. Sequential denoising avoids filtering a large number of frames by decomposing multiple frames denoising into several efficient sub-network denoising. As for each sub-network, we propose an efficient multi-frequency denoising network to remove noise of different frequencies. Our three-stage design is efficient and shows strong performance on burst denoising. Experiments on synthetic and real raw datasets demonstrate that our method outperforms state-of-the-art methods, with less computational cost. Furthermore, the low complexity and high-quality performance make deployment on smartphones possible.Comment: Accepted for publication in International Journal of Computer Visio

A numerical study of the alpha model for two-dimensional magnetohydrodynamic turbulent flows

We explore some consequences of the ``alpha model,'' also called the ``Lagrangian-averaged'' model, for two-dimensional incompressible magnetohydrodynamic (MHD) turbulence. This model is an extension of the smoothing procedure in fluid dynamics which filters velocity fields locally while leaving their associated vorticities unsmoothed, and has proved useful for high Reynolds number turbulence computations. We consider several known effects (selective decay, dynamic alignment, inverse cascades, and the probability distribution functions of fluctuating turbulent quantities) in magnetofluid turbulence and compare the results of numerical solutions of the primitive MHD equations with their alpha-model counterparts' performance for the same flows, in regimes where available resolution is adequate to explore both. The hope is to justify the use of the alpha model in regimes that lie outside currently available resolution, as will be the case in particular in three-dimensional geometry or for magnetic Prandtl numbers differing significantly from unity. We focus our investigation, using direct numerical simulations with a standard and fully parallelized pseudo-spectral method and periodic boundary conditions in two space dimensions, on the role that such a modeling of the small scales using the Lagrangian-averaged framework plays in the large-scale dynamics of MHD turbulence. Several flows are examined, and for all of them one can conclude that the statistical properties of the large-scale spectra are recovered, whereas small-scale detailed phase information (such as e.g. the location of structures) is lost.Comment: 22 pages, 20 figure