Computational imaging and automated identification for aqueous environments

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2011Sampling the vast volumes of the ocean requires tools capable of observing from a distance while retaining detail necessary for biology and ecology, ideal for optical methods. Algorithms that work with existing SeaBED AUV imagery are developed, including habitat classi fication with bag-of-words models and multi-stage boosting for rock sh detection. Methods for extracting images of sh from videos of longline operations are demonstrated. A prototype digital holographic imaging device is designed and tested for quantitative in situ microscale imaging. Theory to support the device is developed, including particle noise and the effects of motion. A Wigner-domain model provides optimal settings and optical limits for spherical and planar holographic references. Algorithms to extract the information from real-world digital holograms are created. Focus metrics are discussed, including a novel focus detector using local Zernike moments. Two methods for estimating lateral positions of objects in holograms without reconstruction are presented by extending a summation kernel to spherical references and using a local frequency signature from a Riesz transform. A new metric for quickly estimating object depths without reconstruction is proposed and tested. An example application, quantifying oil droplet size distributions in an underwater plume, demonstrates the efficacy of the prototype and algorithms.Funding was provided by NOAA Grant #5710002014, NOAA NMFS Grant #NA17RJ1223, NSF Grant #OCE-0925284, and NOAA Grant #NA10OAR417008

Clasificación Tisular en GPU: aceleración y optimizaciones

Desde hace una década, los procesadores gráficos o GPUs vienen ganando protagonismo en la computación de altas prestaciones, contribuyendo a la aceleración de miles de aplicaciones en multitud de áreas de la ciencia. Pero más que esta conquista, lo que ha hecho singular al movimiento GPGPU ha sido la vía para su consecución, ofreciendo tecnología popular, barata y notablemente arropada. Como resultado, la supercomputación está hoy al alcance de cualquier usuario y empresa, democratizando un sector hasta entonces circunscrito a unos pocos centros elitistas. El auge de las GPUs en los entornos de altas prestaciones ha generado un reto a la comunidad de desarrolladores software. Los programadores están habituados a pensar y programar de manera secuencial, y sólo una minoría se atrevía hace 10 años a adentrarse en este mundo. La programación paralela es una tarea compleja que exige otras habilidades y modelo de razonamiento, además de conocer nuevos conceptos hardware, algoritmos y herramientas de programación. Poco a poco, esta percepción ha ido cambiando gracias a la aportación de aquellos que, conscientes de la dificultad, quisieron aportar su granito de arena para facilitar esta transición. El trabajo de esta tesis recoge este espíritu. Planteamos nuevos diseños e implementaciones de algoritmos en el ámbito de la biocomputación para evaluar el rendimiento de las GPUs más destacadas durante la última década, desde equipos con una única GPU hasta supercomputadores de 32 GPUs. En cada uno de los problemas de biocomputación se han analizado todas las características relevantes de la GPU que permiten exprimir su gran potencial, para así presentar de una manera didáctica y rigurosa un estudio pormenorizado de los detalles y técnicas de programación más acordes a cada tipo de algoritmo. Cronológicamente, la aparición de la arquitectura de cálculo paralelo CUDA para GPUs es un hito de especial importancia en la programación de algoritmos de propósito general en GPUs. Nuestro trabajo comenzó en la era pre-CUDA con una aplicación de detección de círculos basada en la transformada de Hough y un algoritmo de detección del tumor neuroblastoma. Sus implementaciones explotan la GPU desde una perspectiva más artesanal, empleando un gran abanico de unidades funcionales de la GPU. Para ello fueron necesarios buenos conocimientos del cauce de segmentación gráfico y ciertas dosis de creatividad. Lo habitual en aquella época era aprovechar casi de forma exclusiva los procesadores de píxeles, al ser los más numerosos y mostrar ya claros indicios de escalabilidad. Entre tanto, nuestro estudio se dedicó a mostrar el potencial de otros recursos menos populares, como los procesadores de vértices, el rasterizador (conversión de polígonos en píxeles) y las unidades de blending (mezclado de contenidos en pantalla). Con la irrupción de CUDA, nuestra atención se dirigió a aplicaciones más exigentes, como el registro de imágenes o el cálculo de los momentos de Zernike para caracterizar regiones tisulares. Completamos también nuestro estudio del neuroblastoma, para poder así contrastar las facilidades aportadas por CUDA y sus posibilidades de optimización. Respecto a las arquitecturas gráficas objeto de nuestro análisis, comenzamos nuestra andadura con modestas GeForce, prosiguiendo con Quadro de gama alta, y concluyendo con Tesla de propósito general, justo donde muchos se iniciaron en el mundo GPGPU para tomar el relevo. La longevidad del algoritmo de detección de tumores nos ha permitido comparar evolutivamente todas estas arquitecturas, el registro de imágenes, ilustrar el beneficio de apoyarse en una popular librería como cuFFT, y los momentos de Zernike, desvelar las exigencias para optimizar el código en generaciones venideras (en nuestro caso, Fermi y Kepler). La exploración de este amplio abanico de posibilidades, tanto en la vertiente software como en la diversidad de modelos hardware que nos han acompañado, desemboca en un sinfín de aportaciones que, además de contribuir a una aceleración de hasta dos órdenes de magnitud en comparación con CPUs de su misma gama, han permitido que el trabajo de esta tesis siente las bases de otras muchas líneas de investigación que han dado crédito y continuidad a nuestro esfuerzo

Towards Predictive Rendering in Virtual Reality

The strive for generating predictive images, i.e., images representing radiometrically correct renditions of reality, has been a longstanding problem in computer graphics. The exactness of such images is extremely important for Virtual Reality applications like Virtual Prototyping, where users need to make decisions impacting large investments based on the simulated images. Unfortunately, generation of predictive imagery is still an unsolved problem due to manifold reasons, especially if real-time restrictions apply. First, existing scenes used for rendering are not modeled accurately enough to create predictive images. Second, even with huge computational efforts existing rendering algorithms are not able to produce radiometrically correct images. Third, current display devices need to convert rendered images into some low-dimensional color space, which prohibits display of radiometrically correct images. Overcoming these limitations is the focus of current state-of-the-art research. This thesis also contributes to this task. First, it briefly introduces the necessary background and identifies the steps required for real-time predictive image generation. Then, existing techniques targeting these steps are presented and their limitations are pointed out. To solve some of the remaining problems, novel techniques are proposed. They cover various steps in the predictive image generation process, ranging from accurate scene modeling over efficient data representation to high-quality, real-time rendering. A special focus of this thesis lays on real-time generation of predictive images using bidirectional texture functions (BTFs), i.e., very accurate representations for spatially varying surface materials. The techniques proposed by this thesis enable efficient handling of BTFs by compressing the huge amount of data contained in this material representation, applying them to geometric surfaces using texture and BTF synthesis techniques, and rendering BTF covered objects in real-time. Further approaches proposed in this thesis target inclusion of real-time global illumination effects or more efficient rendering using novel level-of-detail representations for geometric objects. Finally, this thesis assesses the rendering quality achievable with BTF materials, indicating a significant increase in realism but also confirming the remainder of problems to be solved to achieve truly predictive image generation

Computational imaging and automated identification for aqueous environments

Thesis (Ph. D.)--Joint Program in Oceanography/Applied Ocean Science and Engineering (Massachusetts Institute of Technology, Dept. of Mechanical Engineering; and the Woods Hole Oceanographic Institution), 2011."June 2011." Cataloged from PDF version of thesis.Includes bibliographical references (p. 253-293).Sampling the vast volumes of the ocean requires tools capable of observing from a distance while retaining detail necessary for biology and ecology, ideal for optical methods. Algorithms that work with existing SeaBED AUV imagery are developed, including habitat classification with bag-of-words models and multi-stage boosting for rock sh detection. Methods for extracting images of sh from videos of long-line operations are demonstrated. A prototype digital holographic imaging device is designed and tested for quantitative in situ microscale imaging. Theory to support the device is developed, including particle noise and the effects of motion. A Wigner-domain model provides optimal settings and optical limits for spherical and planar holographic references. Algorithms to extract the information from real-world digital holograms are created. Focus metrics are discussed, including a novel focus detector using local Zernike moments. Two methods for estimating lateral positions of objects in holograms without reconstruction are presented by extending a summation kernel to spherical references and using a local frequency signature from a Riesz transform. A new metric for quickly estimating object depths without reconstruction is proposed and tested. An example application, quantifying oil droplet size distributions in an underwater plume, demonstrates the efficacy of the prototype and algorithms.by Nicholas C. Loomis.Ph.D

Label Efficient 3D Scene Understanding

3D scene understanding models are becoming increasingly integrated into modern society. With applications ranging from autonomous driving, Augmented Real- ity, Virtual Reality, robotics and mapping, the demand for well-behaved models is rapidly increasing. A key requirement for training modern 3D models is high- quality manually labelled training data. Collecting training data is often the time and monetary bottleneck, limiting the size of datasets. As modern data-driven neu- ral networks require very large datasets to achieve good generalisation, finding al- ternative strategies to manual labelling is sought after for many industries. In this thesis, we present a comprehensive study on achieving 3D scene under- standing with fewer labels. Specifically, we evaluate 4 approaches: existing data, synthetic data, weakly-supervised and self-supervised. Existing data looks at the potential of using readily available national mapping data as coarse labels for train- ing a building segmentation model. We further introduce an energy-based active contour snake algorithm to improve label quality by utilising co-registered LiDAR data. This is attractive as whilst the models may still require manual labels, these labels already exist. Synthetic data also exploits already existing data which was not originally designed for training neural networks. We demonstrate a pipeline for generating a synthetic Mobile Laser Scanner dataset. We experimentally evalu- ate if such a synthetic dataset can be used to pre-train smaller real-world datasets, increasing the generalisation with less data. A weakly-supervised approach is presented which allows for competitive per- formance on challenging real-world benchmark 3D scene understanding datasets with up to 95% less data. We propose a novel learning approach where the loss function is learnt. Our key insight is that the loss function is a local function and therefore can be trained with less data on a simpler task. Once trained our loss function can be used to train a 3D object detector using only unlabelled scenes. Our method is both flexible and very scalable, even performing well across datasets. Finally, we propose a method which only requires a single geometric represen- tation of each object class as supervision for 3D monocular object detection. We discuss why typical L2-like losses do not work for 3D object detection when us- ing differentiable renderer-based optimisation. We show that the undesirable local- minimas that the L2-like losses fall into can be avoided with the inclusion of a Generative Adversarial Network-like loss. We achieve state-of-the-art performance on the challenging 6DoF LineMOD dataset, without any scene level labels

Gaze Estimation with Graphics

Gaze estimation systems determine where someone is looking. Gaze is used for a wide range of applications including market research, usability studies, and gaze-based interfaces. Traditional equipment uses special hardware. To bring gaze estimation mainstream, researchers are exploring approaches that use commodity hardware alone. My work addresses two outstanding problems in this field: 1) it is hard to collect good ground truth eye images for machine learning, and 2) gaze estimation systems do not generalize well -- once they are trained with images from one scenario, they do not work in another scenario. In this dissertation I address these problems in two different ways: learning-by-synthesis and analysis-by-synthesis. Learning-by-synthesis is the process of training a machine learning system with synthetic data, i.e. data that has been rendered with graphics rather than collected by hand. Analysis-by-synthesis is a computer vision strategy that couples a generative model of image formation (synthesis) with a perceptive model of scene comparison (analysis). The goal is to synthesize an image that best matches an observed image. In this dissertation I present three main contributions. First, I present a new method for training gaze estimation systems that use machine learning: learning-by-synthesis using 3D head scans and photorealistic rendering. Second, I present a new morphable model of the eye region. I show how this model can be used to generate large amounts of varied data for learning-by-synthesis. Third, I present a new method for gaze estimation: analysis-by-synthesis. I demonstrate how analysis-by-synthesis can generalize to different scenarios, estimating gaze in a device- and person- independent manner.EPSRC Doctoral Training Grant studentship for Erroll Wood (RG71269