Hyperspectral video restoration using optical flow and sparse coding

Hyperspectral video acquisition is a trade-off between spectral and temporal resolution. We present an algorithm for recovering dense hyperspectral video of dynamic scenes from a few measured multispectral bands per frame using optical flow and sparse coding. Different set of bands are measured in each video frame and optical flow is used to register them. Optical flow errors are corrected by exploiting sparsity in the spectra and the spatial correlation between images of a scene at different wavelengths. A redundant dictionary of atoms is learned that can sparsely approximate training spectra. The restoration of correct spectra is formulated as an ℓ1 convex optimization problem that minimizes a Mahalanobis-like weighted distance between the restored and corrupt signals as well as the restored signal and the median of the eight connected neighbours of the corrupt signal such that the restored signal is a sparse linear combination of the dictionary atoms. Spectral restoration is followed by spatial restoration using a guided dictionary approach where one dictionary is learned for measured bands and another for a band that is to be spatially restored. By constraining the sparse coding coefficients of both dictionaries to be the same, the restoration of corrupt band is guided by the more reliable measured bands. Experiments on real data and comparison with an existing volumetric image denoising technique shows the superiority of our algorithm

Light field image processing: an overview

Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data

Efficient Methods for Computational Light Transport

En esta tesis presentamos contribuciones sobre distintos retos computacionales relacionados con transporte de luz. Los algoritmos que utilizan información sobre el transporte de luz están presentes en muchas aplicaciones de hoy en día, desde la generación de efectos visuales, a la detección de objetos en tiempo real. La luz es una valiosa fuente de información que nos permite entender y representar nuestro entorno, pero obtener y procesar esta información presenta muchos desafíos debido a la complejidad de las interacciones entre la luz y la materia. Esta tesis aporta contribuciones en este tema desde dos puntos de vista diferentes: algoritmos en estado estacionario, en los que se asume que la velocidad de la luz es infinita; y algoritmos en estado transitorio, que tratan la luz no solo en el dominio espacial, sino también en el temporal. Nuestras contribuciones en algoritmos estacionarios abordan problemas tanto en renderizado offline como en tiempo real. Nos enfocamos en la reducción de varianza para métodos offline,proponiendo un nuevo método para renderizado eficiente de medios participativos. En renderizado en tiempo real, abordamos las limitacionesde consumo de batería en dispositivos móviles proponiendo un sistema de renderizado que incrementa la eficiencia energética en aplicaciones gráficas en tiempo real. En el transporte de luz transitorio, formalizamos la simulación de este tipo transporte en este nuevo dominio, y presentamos nuevos algoritmos y métodos para muestreo eficiente para render transitorio. Finalmente, demostramos la utilidad de generar datos en este dominio, presentando un nuevo método para corregir interferencia multi-caminos en camaras Timeof- Flight, un problema patológico en el procesamiento de imágenes transitorias.n this thesis we present contributions to different challenges of computational light transport. Light transport algorithms are present in many modern applications, from image generation for visual effects to real-time object detection. Light is a rich source of information that allows us to understand and represent our surroundings, but obtaining and processing this information presents many challenges due to its complex interactions with matter. This thesis provides advances in this subject from two different perspectives: steady-state algorithms, where the speed of light is assumed infinite, and transient-state algorithms, which deal with light as it travels not only through space but also time. Our steady-state contributions address problems in both offline and real-time rendering. We target variance reduction in offline rendering by proposing a new efficient method for participating media rendering. In real-time rendering, we target energy constraints of mobile devices by proposing a power-efficient rendering framework for real-time graphics applications. In transient-state we first formalize light transport simulation under this domain, and present new efficient sampling methods and algorithms for transient rendering. We finally demonstrate the potential of simulated data to correct multipath interference in Time-of-Flight cameras, one of the pathological problems in transient imaging.<br /

Advanced Image Acquisition, Processing Techniques and Applications

"Advanced Image Acquisition, Processing Techniques and Applications" is the first book of a series that provides image processing principles and practical software implementation on a broad range of applications. The book integrates material from leading researchers on Applied Digital Image Acquisition and Processing. An important feature of the book is its emphasis on software tools and scientific computing in order to enhance results and arrive at problem solution

Full-field vibrometry by high-speed digital holography for middle-ear mechanics

Hearing loss affects approximately 1 in 10 people in the world and this percentage is increasing every year. Some of the most common causes of hearing loss are disorders of the middle-ear. Early detection and diagnosis of hearing loss as well as research to understand the hearing processes depend on medical and research tools for quantification of hearing capabilities and the function of the middle-ear in the complex acousto-mechanical transformation of environmental sounds into vibrations of the middle-ear, particular of the human tympanic membrane (TM or eardrum). Current ear exams assess the state of a patient’s hearing capabilities mainly based on qualitative evaluation of the healthiness of the TM. Existing quantitative clinical methods for description of the motion of the TM are limited to either average acoustic estimates (admittance or reflectance) or single-point displacement measurements. Such methods could leave examiners and researchers blind to the complex spatio-temporal response of the nanometer scale displacements of the entire TM. Current state-of-the-art medical research tools provide full-field nanometer displacement measurements of the surface of the human TM excited by steady state (tonal) stimuli. However, to fully understand the mechanics of hearing, and the complex acousto-mechanical characteristics of TM in particular, new tools are needed for full-field high-speed characterization of the nanometer scale displacements of the human TM subjected to impulse (wideband) acoustic excitation. This Dissertation reports the development of a new high-speed holographic system (HHS) for full-field nanometer transient (i.e., \u3e 10 kHz) displacement measurement of the human middle-ear and the tympanic membrane, in particular. The HHS allows spatial (i.e., \u3e500k data points) and temporal (i.e., \u3e 40 kHz) resolutions that enable the study of the acoustical and mechanical characteristics of the middle-ear at a level of detail that have never been reached before. The realization of the HHS includes the development and implementation of novel phase sampling and acquisition approaches that allow the use of state-of-the-art high-resolution (i.e., \u3e5 MP) and high-speed (\u3e 80,000 fps) cameras through modular and expandable control architectures. The development of novel acquisition approaches allows the use of conventional speed (i.e., \u3c20 fps) cameras to realize high-temporal resolutions (i.e., \u3c15 us) at equivalent sampling rates of \u3e 50,000 fps with minimum hardware cost and modifications. The design and implementation of novel spatio-temporal phase sampling methods utilize the high temporal resolution (i.e., \u3c 5 us exposure) and frame rate (i.e., \u3e80,000 fps) of high-speed cameras without imposing constraints on their spatial resolution (i.e., \u3e20 um pixel size). Additionally, the research and in-vivo applications capabilities of the HHS are extended through the development and implementation of a holographic otoscope head (OH) and a mechatronic otoscope positioner (MOP). The large (i.e., \u3e 1 GB with \u3e 8x10^9 parameters) spatio-temporal data sets of the HHS measurements are automatically processed by custom parallel data mining and interpretation (PDMI) methods, which allow automatic quantification of medically relevant motion parameters (MRMPs), such as modal frequencies, time constants, and acoustic delays. Such capabilities could allow inferring local material properties across the surface of the TM. The HHS is a new medical tool that enables otologists to improve the quality of diagnosis and treatments as well as provides researchers with spatio-temporal information of the hearing process at a level of detail never reached before