Advances in Simultaneous Localization and Mapping in Confined Underwater Environments Using Sonar and Optical Imaging.

This thesis reports on the incorporation of surface information into a probabilistic simultaneous localization and mapping (SLAM) framework used on an autonomous underwater vehicle (AUV) designed for underwater inspection. AUVs operating in cluttered underwater environments, such as ship hulls or dams, are commonly equipped with Doppler-based sensors, which---in addition to navigation---provide a sparse representation of the environment in the form of a three-dimensional (3D) point cloud. The goal of this thesis is to develop perceptual algorithms that take full advantage of these sparse observations for correcting navigational drift and building a model of the environment. In particular, we focus on three objectives. First, we introduce a novel representation of this 3D point cloud as collections of planar features arranged in a factor graph. This factor graph representation probabalistically infers the spatial arrangement of each planar segment and can effectively model smooth surfaces (such as a ship hull). Second, we show how this technique can produce 3D models that serve as input to our pipeline that produces the first-ever 3D photomosaics using a two-dimensional (2D) imaging sonar. Finally, we propose a model-assisted bundle adjustment (BA) framework that allows for robust registration between surfaces observed from a Doppler sensor and visual features detected from optical images. Throughout this thesis, we show methods that produce 3D photomosaics using a combination of triangular meshes (derived from our SLAM framework or given a-priori), optical images, and sonar images. Overall, the contributions of this thesis greatly increase the accuracy, reliability, and utility of in-water ship hull inspection with AUVs despite the challenges they face in underwater environments. We provide results using the Hovering Autonomous Underwater Vehicle (HAUV) for autonomous ship hull inspection, which serves as the primary testbed for the algorithms presented in this thesis. The sensor payload of the HAUV consists primarily of: a Doppler velocity log (DVL) for underwater navigation and ranging, monocular and stereo cameras, and---for some applications---an imaging sonar.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120750/1/paulozog_1.pd

Automatic 3d modeling of environments (a sparse approach from images taken by a catadioptric camera)

La modélisation 3d automatique d'un environnement à partir d'images est un sujet toujours d'actualité en vision par ordinateur. Ce problème se résout en général en trois temps : déplacer une caméra dans la scène pour prendre la séquence d'images, reconstruire la géométrie, et utiliser une méthode de stéréo dense pour obtenir une surface de la scène. La seconde étape met en correspondances des points d'intérêts dans les images puis estime simultanément les poses de la caméra et un nuage épars de points 3d de la scène correspondant aux points d'intérêts. La troisième étape utilise l'information sur l'ensemble des pixels pour reconstruire une surface de la scène, par exemple en estimant un nuage de points dense.Ici nous proposons de traiter le problème en calculant directement une surface à partir du nuage épars de points et de son information de visibilité fournis par l'estimation de la géométrie. Les avantages sont des faibles complexités en temps et en espace, ce qui est utile par exemple pour obtenir des modèles compacts de grands environnements comme une ville. Pour cela, nous présentons une méthode de reconstruction de surface du type sculpture dans une triangulation de Delaunay 3d des points reconstruits. L'information de visibilité est utilisée pour classer les tétraèdres en espace vide ou matière. Puis une surface est extraite de sorte à séparer au mieux ces tétraèdres à l'aide d'une méthode gloutonne et d'une minorité de points de Steiner. On impose sur la surface la contrainte de 2-variété pour permettre des traitements ultérieurs classiques tels que lissage, raffinement par optimisation de photo-consistance ... Cette méthode a ensuite été étendue au cas incrémental : à chaque nouvelle image clef sélectionnée dans une vidéo, de nouveaux points 3d et une nouvelle pose sont estimés, puis la surface est mise à jour. La complexité en temps est étudiée dans les deux cas (incrémental ou non). Dans les expériences, nous utilisons une caméra catadioptrique bas coût et obtenons des modèles 3d texturés pour des environnements complets incluant bâtiments, sol, végétation ... Un inconvénient de nos méthodes est que la reconstruction des éléments fins de la scène n'est pas correcte, par exemple les branches des arbres et les pylônes électriques.The automatic 3d modeling of an environment using images is still an active topic in Computer Vision. Standard methods have three steps : moving a camera in the environment to take an image sequence, reconstructing the geometry of the environment, and applying a dense stereo method to obtain a surface model of the environment. In the second step, interest points are detected and matched in images, then camera poses and a sparse cloud of 3d points corresponding to the interest points are simultaneously estimated. In the third step, all pixels of images are used to reconstruct a surface of the environment, e.g. by estimating a dense cloud of 3d points. Here we propose to generate a surface directly from the sparse point cloud and its visibility information provided by the geometry reconstruction step. The advantages are low time and space complexities ; this is useful e.g. for obtaining compact models of large and complete environments like a city. To do so, a surface reconstruction method by sculpting 3d Delaunay triangulation of the reconstructed points is proposed.The visibility information is used to classify the tetrahedra in free-space and matter. Then a surface is extracted thanks to a greedy method and a minority of Steiner points. The 2-manifold constraint is enforced on the surface to allow standard surface post-processing such as denoising, refinement by photo-consistency optimization ... This method is also extended to the incremental case : each time a new key-frame is selected in the input video, new 3d points and camera pose are estimated, then the reconstructed surface is updated.We study the time complexity in both cases (incremental or not). In experiments, a low-cost catadioptric camera is used to generate textured 3d models for complete environments including buildings, ground, vegetation ... A drawback of our methods is that thin scene components cannot be correctly reconstructed, e.g. tree branches and electric posts.CLERMONT FD-Bib.électronique (631139902) / SudocSudocFranceF

Augmented Reality and Artificial Intelligence in Image-Guided and Robot-Assisted Interventions

In minimally invasive orthopedic procedures, the surgeon places wires, screws, and surgical implants through the muscles and bony structures under image guidance. These interventions require alignment of the pre- and intra-operative patient data, the intra-operative scanner, surgical instruments, and the patient. Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. State of the art approaches often support the surgeon by using external navigation systems or ill-conditioned image-based registration methods that both have certain drawbacks. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. Consequently, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. This dissertation investigates the applications of AR, artificial intelligence, and robotics in interventional medicine. Our solutions were applied in a broad spectrum of problems for various tasks, namely improving imaging and acquisition, image computing and analytics for registration and image understanding, and enhancing the interventional visualization. The benefits of these approaches were also discovered in robot-assisted interventions. We revealed how exemplary workflows are redefined via AR by taking full advantage of head-mounted displays when entirely co-registered with the imaging systems and the environment at all times. The proposed AR landscape is enabled by co-localizing the users and the imaging devices via the operating room environment and exploiting all involved frustums to move spatial information between different bodies. The system's awareness of the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. We also leveraged the principles governing image formation and combined it with deep learning and RGBD sensing to fuse images and reconstruct interventional data. We hope that our holistic approaches towards improving the interface of surgery and enhancing the usability of interventional imaging, not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications

Abstracts on Radio Direction Finding (1899 - 1995)

The files on this record represent the various databases that originally composed the CD-ROM issue of "Abstracts on Radio Direction Finding" database, which is now part of the Dudley Knox Library's Abstracts and Selected Full Text Documents on Radio Direction Finding (1899 - 1995) Collection. (See Calhoun record https://calhoun.nps.edu/handle/10945/57364 for further information on this collection and the bibliography). Due to issues of technological obsolescence preventing current and future audiences from accessing the bibliography, DKL exported and converted into the three files on this record the various databases contained in the CD-ROM. The contents of these files are: 1) RDFA_CompleteBibliography_xls.zip [RDFA_CompleteBibliography.xls: Metadata for the complete bibliography, in Excel 97-2003 Workbook format; RDFA_Glossary.xls: Glossary of terms, in Excel 97-2003 Workbookformat; RDFA_Biographies.xls: Biographies of leading figures, in Excel 97-2003 Workbook format]; 2) RDFA_CompleteBibliography_csv.zip [RDFA_CompleteBibliography.TXT: Metadata for the complete bibliography, in CSV format; RDFA_Glossary.TXT: Glossary of terms, in CSV format; RDFA_Biographies.TXT: Biographies of leading figures, in CSV format]; 3) RDFA_CompleteBibliography.pdf: A human readable display of the bibliographic data, as a means of double-checking any possible deviations due to conversion

Endoscopic Fluorescence Imaging:Spectral Optimization and in vivo Characterization of Positive Sites by Magnifying Vascular Imaging

Since several decades, the physicians are able to access hollow organs with endoscopic methods, which serve both as diagnostic and surgical means in a wide range of disciplines of the modern medicine (e.g. urology, pneumology, gastroenterology). Unfortunately, white light (WL) endoscopy displays a limited sensitivity to early pre-cancerous lesions. Hence, several endoscopic methods based on fluorescence imaging have been developed to overcome this limitation. Both endogenous and exogenously-induced fluorescence have been investigated, leading to commercial products. Indeed, autofluorescence bronchoscopy, as well as porphyrin-based fluorescence cystoscopy, are now on the market. As a matter of fact, fluorescence-based endoscopic detection methods show very high sensitivity to pre-cancerous lesions, which are often overlooked in WL endoscopy, but they still lack specificity mainly due to the high false-positive rate. Although most of these false positives can easily be rejected under WL observation, tissue abnormalities such as inflammations, hyperplasia, and metaplasia are more difficult to identify, often resulting in supplementary biopsies. Therefore, the purpose of this thesis is to study novel, fast, and convenient method to characterize fluorescence positive spots in situ during fluorescence endoscopy and, more generally, to optimize the existing endoscopic setup. In this thesis, several clinical evaluations were conducted either in the tracheo-bronchial tree and the urinary bladder. In the urinary bladder, fluorescence imaging for detection of non-muscle invasive bladder cancer is based on the selective production and accumulation of fluorescing porphyrins, mainly protoporphyrin IX (PpIX), in cancerous tissues after the instillation of Hexvix® during one hour. In this thesis, we adapted a rigid cystoscope to perform high magnification (HM) cystoscopy in order to discriminate false from true fluorescence positive findings. Both white light and fluorescence modes are possible with the magnification cystoscope, allowing observation of the bladder wall with magnification ranging between 30× – for standard observation – and 650×. The optical zooming setup allows adjusting the magnification continuously in situ. In the high magnification regime, the smallest diameter of the field of view is 600 microns and the resolution is 2.5 microns, when in contact with the bladder wall. With this HM cystoscope, we characterized the superficial vascularization of the fluorescing sites in WL (370–700 nm) reflectance imaging in order to discriminate cancerous from non-cancerous tissues. This procedure allowed us to establish a classification based on observed vascular patterns. 72 patients subject to Hexvix® f luorescence cystoscopy were included in the study. Comparison of HM cystoscopy classification with histopathology results confirmed 32/33 (97%) cancerous biopsies, and rejected 17/20 (85%) non-cancerous lesions. No vascular alteration could be observed on the only positive lesion that was negative in HM mode, probably because this sarcomatoid carcinoma was not originating in the bladder mucosa. We established with this study that a magnification ranging between 80× and 100× is an optimal tradeoff to perform both macroscopic PDD and HM reflectance imaging. In order to make this approach more quantitative, different algorithms of image processing (vessel segmentation and skeletonisation, global information extraction) were also implemented in this thesis. In order to better visualize the vessels, we improved their contrast with respect to the background. Since hemoglobin is a very strong absorber, we targeted the two hemoglobin absorption peaks by placing appropriate bandpass filters (blue 405±50 nm, green 550±50 nm) in the light source. HM cystoscopy was then performed sequentially with WL, blue and green illumination. The two latter showed higher vessel-to-background contrast, identifying different layers of vascularization due to the light penetration depth. During fluorescence cystoscopy, we often observed that the images are somehow "blurred" by a greenish screen between endoscope tip and bladder mucosa. Since this effect is enhanced by the urine production, it is more visible with flexible scopes (lower flushing capabilities) and imaging systems that collect only autofluorescence as background. Indeed, when the bladder is not flushed regularly, greenish flows coming out of the ureters can easily be observed. For this reason, it is supposed that some fluorophores contained in the urine are excited by the photodetection excitation light, and appear greenish on the screen. This effect may impair the visualization of the bladder mucosa, and thus cancerous lesions, and lowers sensitivity of the fluorescence cystoscopy. In this thesis, we identified the main metabolites responsible for the liquid fluorescence, and optimized the spectral design accordingly. In the tracheo-bronchial tree, the fluorescence contrast is based on the sharp autofluorescence (AF) decrease on early cancerous lesions in the green spectral region (around 500 nm) and a relatively less important decrease in the red spectral region (> 600 nm) when excited with blue-violet light (around 410 nm). It has been shown over the last years, that this contrast may be attributed to a combined effect of epithelium thickening and higher concentration of hemoglobin in the tissues underneath the (pre-)cancerous lesions. In this thesis, we contributed to the definition of the input design of several new prototypes, that were subsequently tested in the clinical environment. We first showed that narrow-band excitation in the blue-violet could increase the tumor-to-normal spectral contrast in the green spectral region. Then, we quantified the intra- and inter-patient variations in the AF intensities in order to optimize the spectral response of the endoscopic fluorescence imaging system. For this purpose, we developed an endoscopic reference to be placed close to the bronchial mucosa during bronchoscopy. Finally, we evaluated a novel AF bronchoscope with blue-backscattered light on 144 patients. This new device showed increased sensitivity for pre-neoplastic lesions. Similar to what we observed in the bladder, it is likely that developing new imaging capabilities (including vascular imaging) will facilitate discriminating true from false positive in AF bronchoscopy. Here, we demonstrated that this magnification allowed us to resolve vessels with a diameter of about 30 µm. This resolution is likely to be sufficient to identify Shibuya's vascular criteria (loops, meshes, dotted vessels) on AF positive lesions. This criteria allow him to recognize pre-cancerous lesions, and thus can potentially decrease the false-positive rate with our AF imaging system. This magnification was also showed to be better for routine bronchoscopy, since it delivers sharper and more structured images to the operator

Manipulador aéreo con brazos antropomórficos de articulaciones flexibles

[Resumen] Este artículo presenta el primer robot manipulador aéreo con dos brazos antropomórficos diseñado para aplicarse en tareas de inspección y mantenimiento en entornos industriales de difícil acceso para operarios humanos. El robot consiste en una plataforma aérea multirrotor equipada con dos brazos antropomórficos ultraligeros, así como el sistema de control integrado de la plataforma y los brazos. Una de las principales características del manipulador es la flexibilidad mecánica proporcionada en todas las articulaciones, lo que aumenta la seguridad en las interacciones físicas con el entorno y la protección del propio robot. Para ello se ha introducido un compacto y simple mecanismo de transmisión por muelle entre el eje del servo y el enlace de salida. La estructura en aluminio de los brazos ha sido cuidadosamente diseñada de forma que los actuadores estén aislados frente a cargas radiales y axiales que los puedan dañar. El manipulador desarrollado ha sido validado a través de experimentos en base fija y en pruebas de vuelo en exteriores.Ministerio de Economía y Competitividad; DPI2014-5983-C2-1-