Computational multi-depth single-photon imaging

We present an imaging framework that is able to accurately reconstruct multiple depths at individual pixels from single-photon observations. Our active imaging method models the single-photon detection statistics from multiple reflectors within a pixel, and it also exploits the fact that a multi-depth profile at each pixel can be expressed as a sparse signal. We interpret the multi-depth reconstruction problem as a sparse deconvolution problem using single-photon observations, create a convex problem through discretization and relaxation, and use a modified iterative shrinkage-thresholding algorithm to efficiently solve for the optimal multi-depth solution. We experimentally demonstrate that the proposed framework is able to accurately reconstruct the depth features of an object that is behind a partially-reflecting scatterer and 4 m away from the imager with root mean-square error of 11 cm, using only 19 signal photon detections per pixel in the presence of moderate background light. In terms of root mean-square error, this is a factor of 4.2 improvement over the conventional method of Gaussian-mixture fitting for multi-depth recovery.This material is based upon work supported in part by a Samsung Scholarship, the US National Science Foundation under Grant No. 1422034, and the MIT Lincoln Laboratory Advanced Concepts Committee. We thank Dheera Venkatraman for his assistance with the experiments. (Samsung Scholarship; 1422034 - US National Science Foundation; MIT Lincoln Laboratory Advanced Concepts Committee)Accepted manuscrip

Seismic Tomography Using Variational Inference Methods

Seismic tomography is a methodology to image the interior of solid or fluid media, and is often used to map properties in the subsurface of the Earth. In order to better interpret the resulting images it is important to assess imaging uncertainties. Since tomography is significantly nonlinear, Monte Carlo sampling methods are often used for this purpose, but they are generally computationally intractable for large datasets and high-dimensional parameter spaces. To extend uncertainty analysis to larger systems we use variational inference methods to conduct seismic tomography. In contrast to Monte Carlo sampling, variational methods solve the Bayesian inference problem as an optimization problem, yet still provide probabilistic results. In this study, we applied two variational methods, automatic differential variational inference (ADVI) and Stein variational gradient descent (SVGD), to 2D seismic tomography problems using both synthetic and real data and we compare the results to those from two different Monte Carlo sampling methods. The results show that variational inference methods can produce accurate approximations to the results of Monte Carlo sampling methods at significantly lower computational cost, provided that gradients of parameters with respect to data can be calculated efficiently. We expect that the methods can be applied fruitfully to many other types of geophysical inverse problems.Comment: 26 pages, 14 figure

Visual Human Tracking and Group Activity Analysis: A Video Mining System for Retail Marketing

Thesis (PhD) - Indiana University, Computer Sciences, 2007In this thesis we present a system for automatic human tracking and activity recognition from video sequences. The problem of automated analysis of visual information in order to derive descriptors of high level human activities has intrigued computer vision community for decades and is considered to be largely unsolved. A part of this interest is derived from the vast range of applications in which such a solution may be useful. We attempt to find efficient formulations of these tasks as applied to the extracting customer behavior information in a retail marketing context. Based on these formulations, we present a system that visually tracks customers in a retail store and performs a number of activity analysis tasks based on the output from the tracker. In tracking we introduce new techniques for pedestrian detection, initialization of the body model and a formulation of the temporal tracking as a global trans-dimensional optimization problem. Initial human detection is addressed by a novel method for head detection, which incorporates the knowledge of the camera projection model.The initialization of the human body model is addressed by newly developed shape and appearance descriptors. Temporal tracking of customer trajectories is performed by employing a human body tracking system designed as a Bayesian jump-diffusion filter. This approach demonstrates the ability to overcome model dimensionality ambiguities as people are leaving and entering the scene. Following the tracking, we developed a two-stage group activity formulation based upon the ideas from swarming research. For modeling purposes, all moving actors in the scene are viewed here as simplistic agents in the swarm. This allows to effectively define a set of inter-agent interactions, which combine to derive a distance metric used in further swarm clustering. This way, in the first stage the shoppers that belong to the same group are identified by deterministically clustering bodies to detect short term events and in the second stage events are post-processed to form clusters of group activities with fuzzy memberships. Quantitative analysis of the tracking subsystem shows an improvement over the state of the art methods, if used under similar conditions. Finally, based on the output from the tracker, the activity recognition procedure achieves over 80% correct shopper group detection, as validated by the human generated ground truth results

Characterisation of the subglacial environment using geophysical constrained Bayesian inversion techniques

An accurate characterization of the inaccessible subglacial environment is key to accurately modelling the dynamic behaviour of ice sheets and glaciers, crucial for predicting sea-level rise. The composition and water content of subglacial material can be inferred from measurements of shear wave velocity (Vs) and bulk electrical resistivity (R), themselves derived from Rayleigh wave dispersion curves and transient electromagnetic (TEM) soundings. Conventional Rayleigh wave and TEM inversions can suffer from poor resolution and non-uniqueness. In this thesis, I present a novel constrained inversion methodology which applies a Markov chain Monte Carlo implementation of Bayesian inversion to produce probability distributions of geophysical parameters. MuLTI (Multimodal Layered Transdimensional Inversion) is used to derive Vs from Rayleigh wave dispersion curves, and its TEM variant, MuLTI-TEM, for evaluating bulk electrical resistivity. The methodologies can include independent depth constraints, drawn from external data sources (e.g., boreholes or other geophysical data), which significantly improves the resolution compared to conventional unconstrained inversions. Compared to such inversions, synthetic studies suggested that MuLTI reduces the error between the true and best-fit models by a factor of 10, and reduces the vertically averaged spread of the Vs distribution twofold, based on the 95% credible intervals. MuLTI and MuLTI-TEM were applied to derive Vs and R profiles from seismic and TEM electromagnetic data acquired on the terminus of the Norwegian glacier Midtdalsbreen. Three subglacial material classifications were determined: sediment (Vs 1600 m/s, R > 500 Ωm) and weathered/fractured bedrock containing saline water (Vs > 1900 m/s, R < 50 Ωm). These algorithms offer a step-change in our ability to resolve and quantify the uncertainties in subsurface inversions, and show promise for constraining the properties of subglacial aquifers beneath Antarctic ice masses. MuLTI and MuLTITEM have both been made publicly available via GitHub to motivate users, in the cryosphere and other environmental settings, for continued advancement

Activity Analysis; Finding Explanations for Sets of Events

Automatic activity recognition is the computational process of analysing visual input and reasoning about detections to understand the performed events. In all but the simplest scenarios, an activity involves multiple interleaved events, some related and others independent. The activity in a car park or at a playground would typically include many events. This research assumes the possible events and any constraints between the events can be defined for the given scene. Analysing the activity should thus recognise a complete and consistent set of events; this is referred to as a global explanation of the activity. By seeking a global explanation that satisfies the activity’s constraints, infeasible interpretations can be avoided, and ambiguous observations may be resolved. An activity’s events and any natural constraints are defined using a grammar formalism. Attribute Multiset Grammars (AMG) are chosen because they allow defining hierarchies, as well as attribute rules and constraints. When used for recognition, detectors are employed to gather a set of detections. Parsing the set of detections by the AMG provides a global explanation. To find the best parse tree given a set of detections, a Bayesian network models the probability distribution over the space of possible parse trees. Heuristic and exhaustive search techniques are proposed to find the maximum a posteriori global explanation. The framework is tested for two activities: the activity in a bicycle rack, and around a building entrance. The first case study involves people locking bicycles onto a bicycle rack and picking them up later. The best global explanation for all detections gathered during the day resolves local ambiguities from occlusion or clutter. Intensive testing on 5 full days proved global analysis achieves higher recognition rates. The second case study tracks people and any objects they are carrying as they enter and exit a building entrance. A complete sequence of the person entering and exiting multiple times is recovered by the global explanation

A Transdimensional Bayesian Approach to Ultrasonic Travel-time Tomography for Non-Destructive Testing

Traditional imaging algorithms within the ultrasonic non-destructive testing community typically assume that the material being inspected is primarily homogeneous, with heterogeneities only at sub-wavelength scales. When the medium is of a more generally heterogeneous nature, this assumption can contribute to the poor detection, sizing and characterisation of any defects. Prior knowledge of the varying velocity fields within the component would allow more accurate imaging of defects, leading to better decisions about how to treat the damaged component. This work endeavours to reconstruct the inhomogeneous velocity fields of random media from simulated ultrasonic phased array data. This is achieved via application of the reversible-jump Markov chain Monte Carlo method: a sampling-based approach within a Bayesian framework. The inverted maps are then used in conjunction with an imaging algorithm to correct for deviations in the wave speed, and the reconstructed flaw images are then used to quantitatively measure the success of this methodology. Using full matrix capture data arising from a finite element simulation of a phased array inspection of a heterogeneous component, a six-fold improvement in flaw location is achieved by taking into account the reconstructed velocity map which exploits almost no \textit{a priori} knowledge of the material's internal structure. Receiver operating characteristic curves are then calculated to demonstrate the enhanced probability of detection achieved when the material map is accounted for

Bayesian inference in seismic tomography

In a variety of scientific applications we require methods to construct three dimensional maps of properties of the interior of solid media, and in the geosciences the medium is usually the Earth's subsurface. For each such map we need the corresponding map of uncertainties in those properties in order to assess their reliability. Seismic tomography is such a method which has been used widely to study properties of the subsurface of the Earth, for example, using surface wave dispersion data. Surface wave tomography is usually conducted using a two-step method by first estimating two-dimensional (2D) surface wave phase or group velocity maps at a series of frequencies and then inverting those for the 3D spatial velocity structure through a set of 1D inversions for structure with depth beneath each geographical location. Since surface wave tomography is a highly non-linear problem, it is usually solved using Monte Carlo (MC) sampling methods. However, since the 1D inversions in the second step are usually performed independently, lateral spatial correlations of the Earth can be lost. We therefore introduce a one-step MC method which inverts for a 3D velocity structure directly from frequency-dependent surface wave travel time measurements by using a fully 3D parametrization. The method was first applied to a synthetic test and compared with two-step linearised and two-step MC methods. The results show that by including lateral spatial correlations in the inversion the new method estimates velocity models and associated uncertainty significantly better in the sense that it produces more intuitively reasonable and interpretable results, and the computation cost is also comparable to the two-step MC method. We apply the 3D MC surface wave tomography method to a real dataset recorded using a dense passive seismic array installed on the North Sea seabed. The ambient noise data of each receiver pair are cross correlated to extract Scholte waves, in which two Scholte wave modes are observed. We separated the two modes using a dispersion compensation method. For each separated mode phase velocity maps are determined using Eikonal tomography. Those phase velocity maps are then used to estimate 3D shear velocities of the subsurface. To further understand the limitation of the approach, we conducted three different inversions: the usual 1D depth inversions, a 2D inversion along a 2D cross section and a fully 3D inversion. With each inversion the shear velocity structure is extracted along the same cross section and compared. The results confirm that 1D inversions can produce errors due to independence of those inversions, whereas 2D and 3D methods improve the results by including lateral spatial correlations in the inversion. The 3D results better match an existing shear velocity model obtained from active source seismic reflection tomography. This is probably because the 3D method uses frequency-dependent measurements directly, which naturally avoids errors introduced in the first 2D Eikonal tomography step. The results show a clear low velocity river channel, and exhibit another low velocity anomaly both in the phase velocity maps at short periods ( < 1.6 s) of the fundamental mode and in the shear-velocity model in the near surface ( < 250 m). The latter anomaly is correlated with the distribution of seabed pockmarks, indicating that the anomaly might be related to the circulation of near surface fluids. Apart from surface waves, seismological body wave travel times have also been used to study the Earth's interior and to characterize earthquakes. Body waves are generally sensitive to structure around the sub-volume in which earthquakes occur and produce limited sensitivity in the near surface, whereas surface waves are more sensitive to the shallower structure. Thus body waves and surface waves can be used jointly to better constrain the subsurface structure. Since the tomographic problem is usually highly non-linear, we apply MC sampling methods to invert for source parameters and velocity models simultaneously using earthquake body wave travel times and ambient noise surface wave dispersion data. The method is applied to a mining site in the U.K. where induced seismicity is recorded using a small local network and ambient noise data are available from the same stations. The results show that by using both types of data, earthquake source parameters and velocity models can be better constrained than in independent inversions. Synthetic tests show that the independent inversion using only body wave travel times can cause biases in the results due to trade-offs between source parameters and velocity models, while this issue can be largely resolved using joint inversion, indicating that the ambient noise data can provide additional information. Although MC sampling methods have been used widely to solve seismic tomographic problems, they are computationally expensive and remain intractable for large dataset problems. We therefore introduce variational inference methods to solve seismic tomographic problems. Variational inference solves the Bayesian inference problem using optimization, yet still provide probabilistic results. In this thesis we introduce two variational methods: automatic differential variational inference (ADVI) and Stein variational gradient descent (SVGD), and apply them to 2D seismic tomographic problems using both synthetic and real data. We compare the results with those obtained using two different MC sampling methods, and demonstrate that variational inference methods can provide accurate approximations to the results of MC sampling methods at significantly lower computational cost, provided that the gradient of model parameters with respect to data can be computed efficiently

Advanced techniques for subsurface imaging Bayesian neural networks and Marchenko methods

Estimation of material properties such as density and velocity of the Earth’s subsurface are important in resource exploration, waste and CO2 storage and for monitoring changes underground. These properties can be used to create structural images of the subsurface or for resource characterisation. Seismic data are often the main source of information from which these estimates are derived. However the complex nature of the Earth, limitations in data acquisition and in resolution of images, and various types of noise all mean that estimates of material parameters also come with a level of uncertainty. The physics relating these material parameters to recorded seismic data is usually non-linear, necessitating the use of Monte Carlo inversion methods to solve the estimation problem in a fully probabilistic sense. Such methods are computationally expensive which usually prohibits their use over areas with many data, or for subsurface models that involve many parameters. Furthermore multiple unknown material parameters can be jointly dependent on each datum so trade-offs between parameters deteriorate parameter estimates and increase uncertainty in the results. In this thesis various types of neural networks are trained to provide probabilistic estimates of the subsurface velocity structure. A trained network can rapidly invert data in near real- time, much more rapidly than any traditional non-linear sampling method such as Monte Carlo. The thesis also shows how the density estimation problem can be reformulated to avoid direct trade-offs with velocity, by using a combination of seismic interferometry and Marchenko methods. First this thesis shows how neural networks can provide a full probability density function describing the uncertainty in parameters of interest, by using a form of network called a mixture density network. This type of network uses a weighted sum of kernel distributions (in our case Gaussians) to model the Bayesian posterior probability density function. The method is demonstrated by inverting localised phase velocity dispersion curves for shear-wave velocity profiles at the scale of a subsurface fluid reservoir, and is applied to field data from the North Sea. This work shows that when the data contain significant noise, including data uncertainties in the network gives more reliable mean velocity estimates. Whilst the post-training inversion process is rapid using neural networks, the method to estimate localised phase velocities in the first place is significantly slower. Therefore a computationally cheap method is demonstrated that combines gradiometry to estimate phase velocities and mixture density networks to invert for subsurface velocity-depth structure, the whole process taking a matter of minutes. This opens the possibility of real-time monitoring using spatially dense surface seismic arrays. For some monitoring situations a dense array is not available and gradiometry therefore cannot be applied to estimate phase velocities. In a third application this thesis uses mixture density networks to invert travel-time data for 2D localised velocity maps with associated uncertainty estimates. The importance of prior information in high dimensional inverse problems is also demonstrated. A new method is then developed to estimate density in the subsurface using a formulation of seismic interferometry that contains a linear dependence of seismic data on subsurface density, avoiding the usual direct trade-off between density and velocity. When wavefields cannot be measured directly in the subsurface, the method requires the use of a technique called Marchenko redatuming that can estimate the Green’s function from a virtual source or receiver inside a medium to the surface. This thesis shows that critical to implementing this work would be the development of more robust methods to scale the amplitude of Green’s function estimates from Marchenko methods. Finally the limitations of the methods presented in this thesis are discussed, as are suggestions for further research, and alternative applications for some of the methods. Overall this thesis proposes several new ways to monitor the subsurface efficiently using probabilistic machine learning techniques, discusses a novel way to estimate subsurface density, and demonstrates the methods on a mixture of synthetic and field data

Generative modeling of dynamic visual scenes

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 301-312).Modeling visual scenes is one of the fundamental tasks of computer vision. Whereas tremendous efforts have been devoted to video analysis in past decades, most prior work focuses on specific tasks, leading to dedicated methods to solve them. This PhD thesis instead aims to derive a probabilistic generative model that coherently integrates different aspects, notably appearance, motion, and the interaction between them. Specifically, this model considers each video as a composite of dynamic layers, each associated with a covering domain, an appearance template, and a flow describing its motion. These layers change dynamically following the associated flows, and are combined into video frames according to a Z-order that specifies their relative depth-order. To describe these layers and their dynamic changes, three major components are incorporated: (1) An appearance model describes the generative process of the pixel values of a video layer. This model, via the combination of a probabilistic patch manifold and a conditional Markov random field, is able to express rich local details while maintaining global coherence. (2) A motion model captures the motion pattern of a layer through a new concept called geometric flow that originates from differential geometric analysis. A geometric flow unifies the trajectory-based representation and the notion of geometric transformation to represent the collective dynamic behaviors persisting over time. (3) A partial Z-order specifies the relative depth order between layers. Here, through the unique correspondence between equivalent classes of partial orders and consistent choice functions, a distribution over the spaces of partial orders is established, and inference can thus be performed thereon. The development of these models leads to significant challenges in probabilistic modeling and inference that need new techniques to address. We studied two important problems: (1) Both the appearance model and the motion model rely on mixture modeling to capture complex distributions. In a dynamic setting, the components parameters and the number of components in a mixture model can change over time. While the use of Dirichlet processes (DPs) as priors allows indefinite number of components, incorporating temporal dependencies between DPs remains a nontrivial issue, theoretically and practically. Our research on this problem leads to a new construction of dependent DPs, enabling various forms of dynamic variations for nonparametric mixture models by harnessing the connections between Poisson and Dirichlet processes. (2) The inference of partial Z-order from a video needs a method to sample from the posterior distribution of partial orders. A key challenge here is that the underlying space of partial orders is disconnected, meaning that one may not be able to make local updates without violating the combinatorial constraints for partial orders. We developed a novel sampling method to tackle this problem, which dynamically introduces virtual states as bridges to connect between different parts of the space, implicitly resulting in an ergodic Markov chain over an augmented space. With this generative model of visual scenes, many vision problems can be readily solved through inference performed on the model. Empirical experiments demonstrate that this framework yields promising results on a series of practical tasks, including video denoising and inpainting, collective motion analysis, and semantic scene understanding.by Dahua Lin.Ph.D

Time-lapse seismic imaging and uncertainty quantification

Time–lapse (4D) seismic monitoring is to date the most commonly used technique for estimating changes of a reservoir under production. Full–Waveform Inversion (FWI) is a high resolution technique that delivers Earth models by iteratively trying to match synthetic prestack seismic data with the observed data. Over the past decade the application of FWI on 4D data has been extensively studied, with a variety of strategies being currently available. However, 4D FWI still has challenges unsolved. In addition, the standard outcome of a 4D FWI scheme is a single image, without any measurement of the associated uncertainty. These issues beg the following questions: (1) Can we go beyond the current FWI limitations and deliver more accurate 4D imaging?, and (2) How well do we know what we think we know? In this thesis, I take steps to answer both questions. I first compare the performances of three common 4D FWI approaches in the presence of model uncertainties. These results provide a preliminary understanding of the underlying uncertainty, but also highlight some of the limitations of pixel by pixel uncertainty quantification. I then introduce a hybrid inversion technique that I call Dual–Domain Waveform Inversion (DDWI), whose objective function joins traditional FWI with Image Domain Wavefield Tomography (IDWT). The new objective function combines diving wave information in the data–domain FWI term with reflected wave information in the image–domain IDWT term, resulting in more accurate 4D model reconstructions. Working with 4D data provides an ideal situation for testing and developing new algorithms. Since there are repeated surveys at the same location, not only is the surrounding geology well–known and the results of interest are localized in small regions, but also they allow for better error analysis. Uncertainty quantification is very valuable for building knowledge but is not commonly done due to the computational challenge of exploring the range of all possible models that could fit the data. I exploit the structure of the 4D problem and propose the use of a focused modeling technique for a fast Metropolis–Hastings inversion. The proposed framework calculates time–lapse uncertainty quantification in a targeted way that is computationally feasible. Having the ground truth 4D probability distributions, I propose a local 4D Hamiltonian Monte Carlo (HMC) — a more advanced uncertainty quantification technique — that can handle higher dimensionalities while offering faster convergence