Motion connected operators for image sequences

This paper deals with motion-oriented connected operators. These operators eliminate from an original sequence the components that do not undergo a specific motion (defined as a filtering parameter). As any connected operator, they achieve a simplification of the original image while preserving the contour information of the components that have not be removed. Motion-oriented filtering may have a large number of applications including sequence analysis with motion multi-resolution decomposition or motion estimation.Peer ReviewedPostprint (published version

Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences

Machine learning based Single Image Intrinsic Decomposition (SIID) methods decompose a captured scene into its albedo and shading images by using the knowledge of a large set of known and realistic ground truth decompositions. Collecting and annotating such a dataset is an approach that cannot scale to sufficient variety and realism. We free ourselves from this limitation by training on unannotated images. Our method leverages the observation that two images of the same scene but with different lighting provide useful information on their intrinsic properties: by definition, albedo is invariant to lighting conditions, and cross-combining the estimated albedo of a first image with the estimated shading of a second one should lead back to the second one's input image. We transcribe this relationship into a siamese training scheme for a deep convolutional neural network that decomposes a single image into albedo and shading. The siamese setting allows us to introduce a new loss function including such cross-combinations, and to train solely on (time-lapse) images, discarding the need for any ground truth annotations. As a result, our method has the good properties of i) taking advantage of the time-varying information of image sequences in the (pre-computed) training step, ii) not requiring ground truth data to train on, and iii) being able to decompose single images of unseen scenes at runtime. To demonstrate and evaluate our work, we additionally propose a new rendered dataset containing illumination-varying scenes and a set of quantitative metrics to evaluate SIID algorithms. Despite its unsupervised nature, our results compete with state of the art methods, including supervised and non data-driven methods.Comment: To appear in Pacific Graphics 201

Time-sequential Pipelined Imaging with Wavefront Coding and Super Resolution

Wavefront coding has long offered the prospect of mitigating optical aberrations and extended depth of field, but image quality and noise performance are inevitably reduced. We report on progress in the use of agile encoding and pipelined fusion of image sequences to recover image quality

Topological Tracking of Connected Components in Image Sequences

Persistent homology provides information about the lifetime of homology classes along a filtration of cell complexes. Persistence barcode is a graphical representation of such information. A filtration might be determined by time in a set of spatiotemporal data, but classical methods for computing persistent homology do not respect the fact that we can not move backwards in time. In this paper, taking as input a time-varying sequence of two-dimensional (2D) binary digital images, we develop an algorithm for encoding, in the so-called {\it spatiotemporal barcode}, lifetime of connected components (of either the foreground or background) that are moving in the image sequence over time (this information may not coincide with the one provided by the persistence barcode). This way, given a connected component at a specific time in the sequence, we can track the component backwards in time until the moment it was born, by what we call a {\it spatiotemporal path}. The main contribution of this paper with respect to our previous works lies in a new algorithm that computes spatiotemporal paths directly, valid for both foreground and background and developed in a general context, setting the ground for a future extension for tracking higher dimensional topological features in $nD$ binary digital image sequences

Laryngoscopic Image Stitching for View Enhancement and Documentation - First Experiences

One known problem within laryngoscopy is the spatially limited view onto the hypopharynx and the larynx through the endoscope. To examine the complete larynx and hypopharynx, the laryngoscope can be rotated about its main axis, and hence the physician obtains a complete view. If such examinations are captured using endoscopic video, the examination can be reviewed in detail at a later time. Nevertheless, in order to document the examination with a single representative image, a panorama image can be computed for archiving and enhanced documentation. Twenty patients with various clinical findings were examined with a 70 rigid laryngoscope, and the video sequences were digitally stored. The image sequence for each patient was then post-processed using an image stitching tool based on SIFT features, the RANSAC approach and blending. As a result, endoscopic panorama images of the larynx and pharynx were obtained for each video sequence. The proposed approach of image stitching for laryngoscopic video sequences offers a new tool for enhanced visual examination and documentation of morphologic characteristics of the larynx and the hypopharynx

Recursive Estimation of Camera Motion from Uncalibrated Image Sequences

In This memo we present an extension of the motion estimation scheme presented in a previous CDS technical report [14, 16], in order to deal with image sequences coming from an uncalibrated camera. The scheme is based on some results in epipolar geometry and invariant theory which can be found in [6]. Experiments are performed on noisy synthetic images