3D Reconstruction of Indoor Corridor Models Using Single Imagery and Video Sequences

In recent years, 3D indoor modeling has gained more attention due to its role in decision-making process of maintaining the status and managing the security of building indoor spaces. In this thesis, the problem of continuous indoor corridor space modeling has been tackled through two approaches. The first approach develops a modeling method based on middle-level perceptual organization. The second approach develops a visual Simultaneous Localisation and Mapping (SLAM) system with model-based loop closure. In the first approach, the image space was searched for a corridor layout that can be converted into a geometrically accurate 3D model. Manhattan rule assumption was adopted, and indoor corridor layout hypotheses were generated through a random rule-based intersection of image physical line segments and virtual rays of orthogonal vanishing points. Volumetric reasoning, correspondences to physical edges, orientation map and geometric context of an image are all considered for scoring layout hypotheses. This approach provides physically plausible solutions while facing objects or occlusions in a corridor scene. In the second approach, Layout SLAM is introduced. Layout SLAM performs camera localization while maps layout corners and normal point features in 3D space. Here, a new feature matching cost function was proposed considering both local and global context information. In addition, a rotation compensation variable makes Layout SLAM robust against cameras orientation errors accumulations. Moreover, layout model matching of keyframes insures accurate loop closures that prevent miss-association of newly visited landmarks to previously visited scene parts. The comparison of generated single image-based 3D models to ground truth models showed that average ratio differences in widths, heights and lengths were 1.8%, 3.7% and 19.2% respectively. Moreover, Layout SLAM performed with the maximum absolute trajectory error of 2.4m in position and 8.2 degree in orientation for approximately 318m path on RAWSEEDS data set. Loop closing was strongly performed for Layout SLAM and provided 3D indoor corridor layouts with less than 1.05m displacement errors in length and less than 20cm in width and height for approximately 315m path on York University data set. The proposed methods can successfully generate 3D indoor corridor models compared to their major counterpart

Multi Camera Stereo and Tracking Patient Motion for SPECT Scanning Systems

Patient motion, which causes artifacts in reconstructed images, can be a serious problem in Single Photon Emission Computed Tomography (SPECT) imaging. If patient motion can be detected and quantified, the reconstruction algorithm can compensate for the motion. A real-time multi-threaded Visual Tracking System (VTS) using optical cameras, which will be suitable for deployment in clinical trials, is under development. The VTS tracks patients using multiple video images and image processing techniques, calculating patient motion in three-dimensional space. This research aimed to develop and implement an algorithm for feature matching and stereo location computation using multiple cameras. Feature matching is done based on the epipolar geometry constraints for a pair of images and extended to the multiple view case with an iterative algorithm. Stereo locations of the matches are then computed using sum of squared distances from the projected 3D lines in SPECT coordinates as the error metric. This information from the VTS, when coupled with motion assessment from the emission data itself, can provide a robust compensation for patient motion as part of reconstruction

Exploiting Structural Regularities and Beyond: Vision-based Localization and Mapping in Man-Made Environments

Image-based estimation of camera motion, known as visual odometry (VO), plays a very important role in many robotic applications such as control and navigation of unmanned mobile robots, especially when no external navigation reference signal is available. The core problem of VO is the estimation of the camera’s ego-motion (i.e. tracking) either between successive frames, namely relative pose estimation, or with respect to a global map, namely absolute pose estimation. This thesis aims to develop efficient, accurate and robust VO solutions by taking advantage of structural regularities in man-made environments, such as piece-wise planar structures, Manhattan World and more generally, contours and edges. Furthermore, to handle challenging scenarios that are beyond the limits of classical sensor based VO solutions, we investigate a recently emerging sensor — the event camera and study on event-based mapping — one of the key problems in the event-based VO/SLAM. The main achievements are summarized as follows. First, we revisit an old topic on relative pose estimation: accurately and robustly estimating the fundamental matrix given a collection of independently estimated homograhies. Three classical methods are reviewed and then we show a simple but nontrivial two-step normalization within the direct linear method that achieves similar performance to the less attractive and more computationally intensive hallucinated points based method. Second, an efficient 3D rotation estimation algorithm for depth cameras in piece-wise planar environments is presented. It shows that by using surface normal vectors as an input, planar modes in the corresponding density distribution function can be discovered and continuously tracked using efficient non-parametric estimation techniques. The relative rotation can be estimated by registering entire bundles of planar modes by using robust L1-norm minimization. Third, an efficient alternative to the iterative closest point algorithm for real-time tracking of modern depth cameras in ManhattanWorlds is developed. We exploit the common orthogonal structure of man-made environments in order to decouple the estimation of the rotation and the three degrees of freedom of the translation. The derived camera orientation is absolute and thus free of long-term drift, which in turn benefits the accuracy of the translation estimation as well. Fourth, we look into a more general structural regularity—edges. A real-time VO system that uses Canny edges is proposed for RGB-D cameras. Two novel alternatives to classical distance transforms are developed with great properties that significantly improve the classical Euclidean distance field based methods in terms of efficiency, accuracy and robustness. Finally, to deal with challenging scenarios that go beyond what standard RGB/RGB-D cameras can handle, we investigate the recently emerging event camera and focus on the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping

Magnetic Resonance Imaging of Short-T2 Tissues with Applications for Quantifying Cortical Bone Water and Myelin

The human body contains a variety of tissue species with short T2 ranging from a few microseconds to hundreds of microseconds. Detection and quantification of these short-T2 species is of considerable clinical and scientific interest. Cortical bone water and myelin are two of the most important tissue constituents. Quantification of cortical bone water concentration allows for indirect estimation of bone pore volume and noninvasive assessment of bone quality. Myelin is essential for the proper functioning of the central nervous system (CNS). Direct assessment of myelin would reveal CNS abnormalities and enhance our understanding of neurological diseases. However, conventional MRI with echo times of several milliseconds or longer is unable to detect these short-lived MR signals. Recent advances in MRI technology and hardware have enabled development of a number of short-T2 imaging techniques, key among which are ultra-short echo time (UTE) imaging, zero echo time (ZTE) imaging, and sweep imaging with Fourier transform (SWIFT). While these pulse sequences are able to detect short-T2 species, they still suffer from signal interference between different T2 tissue constituents, image artifacts and excessive scan time. These are primary technical hurdles for application to whole-body clinical scanners. In this thesis research, new MRI techniques for improving short-T2 tissue imaging have been developed to address these challenges with a focus on direct detection and quantification of cortical bone water and myelin on a clinical MRI scanner. The first focus of this research was to optimize long-T2 suppression in UTE imaging. Saturation and adiabatic RF pulses were designed to achieve maximum long-T2 suppression while maximizing the signal from short-T2 species. The imaging protocols were optimized by Bloch equation simulations and were validated using phantom and in vivo experiments. The results show excellent short-T2 contrast with these optimized pulse sequences. The problem of blurring artifacts resulting from the inhomogeneous excitation profile of the rectangular pulses in ZTE imaging was addressed. The proposed approach involves quadratic phase-modulated RF excitation and iterative solution of an inverse problem formulated from the signal model of ZTE imaging and is shown to effectively remove the image artifacts. Subsequently image acquisition efficiency was improved in order to attain clinically-feasible scan times. To accelerate the acquisition speed in UTE and ZTE imaging, compressed sensing was applied with a hybrid 3D UTE sequence. Further, the pulse sequence and reconstruction procedure were modified to enable anisotropic field-of-view shape conforming to the geometry of the elongated imaged object. These enhanced acquisition techniques were applied to the detection and quantification of cortical bone water. A new biomarker, the suppression ratio (a ratio image derived from two UTE images, one without and the other with long-T2 suppression), was conceived as a surrogate measure of cortical bone porosity. Experimental data suggest the suppression ratio may be a more direct measure of porosity than previously measured total bone water concentration. Lastly, the feasibility of directly detecting and quantifying spatially-resolved myelin concentration with a clinical imager was explored, both theoretically and experimentally. Bloch equation simulations were conducted to investigate the intrinsic image resolution and the fraction of detectable myelin signal under current scanner hardware constraints. The feasibility of quantitative ZTE imaging of myelin extract and lamb spinal cord at 3T was demonstrated. The technological advances achieved in this dissertation research may facilitate translation of short-T2 MRI methods from the laboratory to the clinic

Monocular slam for deformable scenarios.

El problema de localizar la posición de un sensor en un mapa incierto que se estima simultáneamente se conoce como Localización y Mapeo Simultáneo --SLAM--. Es un problema desafiante comparable al paradigma del huevo y la gallina. Para ubicar el sensor necesitamos conocer el mapa, pero para construir el mapa, necesitamos la posición del sensor. Cuando se utiliza un sensor visual, por ejemplo, una cámara, se denomina Visual SLAM o VSLAM. Los sensores visuales para SLAM se dividen entre los que proporcionan información de profundidad (por ejemplo, cámaras RGB-D o equipos estéreo) y los que no (por ejemplo, cámaras monoculares o cámaras de eventos). En esta tesis hemos centrado nuestra investigación en SLAM con cámaras monoculares.Debido a la falta de percepción de profundidad, el SLAM monocular es intrínsecamente más duro en comparación con el SLAM con sensores de profundidad. Los trabajos estado del arte en VSLAM monocular han asumido normalmente que la escena permanece rígida durante toda la secuencia, lo que es una suposición factible para entornos industriales y urbanos. El supuesto de rigidez aporta las restricciones suficientes al problema y permite reconstruir un mapa fiable tras procesar varias imágenes. En los últimos años, el interés por el SLAM ha llegado a las áreas médicas donde los algoritmos SLAM podrían ayudar a orientar al cirujano o localizar la posición de un robot. Sin embargo, a diferencia de los escenarios industriales o urbanos, en secuencias dentro del cuerpo, todo puede deformarse eventualmente y la suposición de rigidez acaba siendo inválida en la práctica, y por extensión, también los algoritmos de SLAM monoculares. Por lo tanto, nuestro objetivo es ampliar los límites de los algoritmos de SLAM y concebir el primer sistema SLAM monocular capaz de hacer frente a la deformación de la escena.Los sistemas de SLAM actuales calculan la posición de la cámara y la estructura del mapa en dos subprocesos concurrentes: la localización y el mapeo. La localización se encarga de procesar cada imagen para ubicar el sensor de forma continua, en cambio el mapeo se encarga de construir el mapa de la escena. Nosotros hemos adoptado esta estructura y concebimos tanto la localización deformable como el mapeo deformable ahora capaces de recuperar la escena incluso con deformación.Nuestra primera contribución es la localización deformable. La localización deformable utiliza la estructura del mapa para recuperar la pose de la cámara con una única imagen. Simultáneamente, a medida que el mapa se deforma durante la secuencia, también recupera la deformación del mapa para cada fotograma. Hemos propuesto dos familias de localización deformable. En el primer algoritmo de localización deformable, asumimos que todos los puntos están embebidos en una superficie denominada plantilla. Podemos recuperar la deformación de la superficie gracias a un modelo de deformación global que permite estimar la deformación más probable del objeto. Con nuestro segundo algoritmo de localización deformable, demostramos que es posible recuperar la deformación del mapa sin un modelo de deformación global, representando el mapa como surfels individuales. Nuestros resultados experimentales mostraron que, recuperando la deformación del mapa, ambos métodos superan tanto en robustez como en precisión a los métodos rígidos.Nuestra segunda contribución es la concepción del mapeo deformable. Es el back-end del algoritmo SLAM y procesa un lote de imágenes para recuperar la estructura del mapa para todas las imágenes y hacer crecer el mapa ensamblando las observaciones parciales del mismo. Tanto la localización deformable como el mapeo que se ejecutan en paralelo y juntos ensamblan el primer SLAM monocular deformable: \emph{DefSLAM}. Una evaluación ampliada de nuestro método demostró, tanto en secuencias controladas por laboratorio como en secuencias médicas, que nuestro método procesa con éxito secuencias en las que falla el sistema monocular SLAM actual.Nuestra tercera contribución son dos métodos para explotar la información fotométrica en SLAM monocular deformable. Por un lado, SD-DefSLAM que aprovecha el emparejamiento semi-directo para obtener un emparejamiento mucho más fiable de los puntos del mapa en las nuevas imágenes, como consecuencia, se demostró que es más robusto y estable en secuencias médicas. Por otro lado, proponemos un método de Localización Deformable Directa y Dispersa en el que usamos un error fotométrico directo para rastrear la deformación de un mapa modelado como un conjunto de surfels 3D desconectados. Podemos recuperar la deformación de múltiples superficies desconectadas, deformaciones no isométricas o superficies con una topología cambiante.<br /

Functional Brain Organization in Space and Time

The brain is a network functionally organized at many spatial and temporal scales. To understand how the brain processes information, controls behavior and dynamically adapts to an ever-changing environment, it is critical to have a comprehensive description of the constituent elements of this network and how relationships between these elements may change over time. Decades of lesion studies, anatomical tract-tracing, and electrophysiological recording have given insight into this functional organization. Recently, however, resting state functional magnetic resonance imaging (fMRI) has emerged as a powerful tool for whole-brain non-invasive measurement of spontaneous neural activity in humans, giving ready access to macroscopic scales of functional organization previously much more difficult to obtain. This thesis aims to harness the unique combination of spatial and temporal resolution provided by functional MRI to explore the spatial and temporal properties of the functional organization of the brain. First, we establish an approach for defining cortical areas using transitions in correlated patterns of spontaneous BOLD activity (Chapter 2). We then propose and apply measures of internal and external validity to evaluate the credibility of the areal parcellation generated by this technique (Chapter 3). In chapter 4, we extend the study of functional brain organization to a highly sampled individual. We describe the idiosyncratic areal and systems-level organization of the individual relative to a standard group-average description. Further, we develop a model describing the reliability of BOLD correlation estimates across days that accounts for relevant sources of variability. Finally, in Chapter 5, we examine whether BOLD correlations meaningfully vary over the course of single resting-state scans

Analysis of image noise in multispectral color acquisition

The design of a system for multispectral image capture will be influenced by the imaging application, such as image archiving, vision research, illuminant modification or improved (trichromatic) color reproduction. A key aspect of the system performance is the effect of noise, or error, when acquiring multiple color image records and processing of the data. This research provides an analysis that allows the prediction of the image-noise characteristics of systems for the capture of multispectral images. The effects of both detector noise and image processing quantization on the color information are considered, as is the correlation between the errors in the component signals. The above multivariate error-propagation analysis is then applied to an actual prototype system. Sources of image noise in both digital camera and image processing are related to colorimetric errors. Recommendations for detector characteristics and image processing for future systems are then discussed