Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy

Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syndrome consists in the photo-coagulation of undesired anastomoses located on the placenta which are responsible to a blood transfer between the two twins. While being the standard of care procedure, fetoscopy suffers from a limited field-of-view of the placenta resulting in missed anastomoses. To facilitate the task of the clinician, building a global map of the placenta providing a larger overview of the vascular network is highly desired. Methods: To overcome the challenging visual conditions inherent to in vivo sequences (low contrast, obstructions or presence of artifacts, among others), we propose the following contributions: (i) robust pairwise registration is achieved by aligning the orientation of the image gradients, and (ii) difficulties regarding long-range consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word strategy, which identifies overlapping frames of the sequence to be registered regardless of their respective location in time. Results: In addition to visual difficulties, in vivo sequences are characterised by the intrinsic absence of gold standard. We present mosaics motivating qualitatively our methodological choices and demonstrating their promising aspect. We also demonstrate semi-quantitatively, via visual inspection of registration results, the efficacy of our registration approach in comparison to two standard baselines. Conclusion: This paper proposes the first approach for the construction of mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual challenges during registration and long-range temporal consistency are proposed, offering first positive results on in vivo data for which standard mosaicking techniques are not applicable.Comment: Accepted for publication in International Journal of Computer Assisted Radiology and Surgery (IJCARS

ROAM: a Rich Object Appearance Model with Application to Rotoscoping

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest. Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given a first closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling

The SSDC contribution to the improvement of knowledge by means of 3D data projections of minor bodies

The latest developments of planetary exploration missions devoted to minor bodies required new solutions to correctly visualize and analyse data acquired over irregularly shaped bodies. ASI Space Science Data Center (SSDC-ASI, formerly ASDC-ASI Science Data Center) worked on this task since early 2013, when started developing the web tool MATISSE (Multi-purpose Advanced Tool for the Instruments of the Solar System Exploration) mainly focused on the Rosetta/ESA space mission data. In order to visualize very high-resolution shape models, MATISSE uses a Python module (vtpMaker), which can also be launched as a stand-alone command-line software. MATISSE and vtpMaker are part of the SSDC contribution to the new challenges imposed by the "orbital exploration" of minor bodies: 1) MATISSE allows to search for specific observations inside datasets and then analyse them in parallel, providing high-level outputs; 2) the 3D capabilities of both tools are critical in inferring information otherwise difficult to retrieve for non-spherical targets and, as in the case for the GIADA instrument onboard Rosetta, to visualize data related to the coma. New tasks and features adding valuable capabilities to the minor bodies SSDC tools are planned for the near future thanks to new collaborations

Content-aware photo collage using circle packing

published_or_final_versio

Seabed fluid flow-related processes: evidence and quantification based on high-resolution imaging techniques and GIS analyses

This work provides new insights on different aspects of seabed fluid flow processes based on seafloor observations. The methods used entirely rely on ROV-based high-resolution imaging and mapping techniques. Optical data are used to produce visual maps of the seafloor, in the form of geo-referenced video- and photo-mosaics, whereas acoustic techniques allow mapping the micro-bathymetry of the seabed, as well as the signal reflectivity of the sediment surface and of the water column. This work presents three case studies, about two sites of seabed fluid flow: the Menez Gwen hydrothermal vent on the MAR and the REGAB pockmark in the Lower Congo Basin. On the technical side, some of the high-resolution techniques used in this thesis are not commonly used by the marine scientific community. This is particularly the case for large-area photo-mosaics. Although the interest in mosaicking is growing, there are still no tools freely and readily available to scientists to routinely construct large-area photo-mosaics. Therefore, this work presents a MATLAB toolbox for large-area photo-mosaicking (LAPM toolbox), which was developed as part of this thesis

Toward autonomous exploration in confined underwater environments

Author Posting. © The Author(s), 2015. This is the author's version of the work. It is posted here by permission of John Wiley & Sons for personal use, not for redistribution. The definitive version was published in Journal of Field Robotics 33 (2016): 994-1012, doi:10.1002/rob.21640.In this field note we detail the operations and discuss the results of an experiment conducted in the unstructured environment of an underwater cave complex, using an autonomous underwater vehicle (AUV). For this experiment the AUV was equipped with two acoustic sonar to simultaneously map the caves’ horizontal and vertical surfaces. Although the caves’ spatial complexity required AUV guidance by a diver, this field deployment successfully demonstrates a scan matching algorithm in a simultaneous localization and mapping (SLAM) framework that significantly reduces and bounds the localization error for fully autonomous navigation. These methods are generalizable for AUV exploration in confined underwater environments where surfacing or pre-deployment of localization equipment are not feasible and may provide a useful step toward AUV utilization as a response tool in confined underwater disaster areas.This research work was partially sponsored by the EU FP7-Projects: Tecniospring- Marie Curie (TECSPR13-1-0052), MORPH (FP7-ICT-2011-7-288704), Eurofleets2 (FP7-INF-2012-312762), and the National Science Foundation (OCE-0955674)