Exploiting Structural Regularities and Beyond: Vision-based Localization and Mapping in Man-Made Environments

Image-based estimation of camera motion, known as visual odometry (VO), plays a very important role in many robotic applications such as control and navigation of unmanned mobile robots, especially when no external navigation reference signal is available. The core problem of VO is the estimation of the camera’s ego-motion (i.e. tracking) either between successive frames, namely relative pose estimation, or with respect to a global map, namely absolute pose estimation. This thesis aims to develop efficient, accurate and robust VO solutions by taking advantage of structural regularities in man-made environments, such as piece-wise planar structures, Manhattan World and more generally, contours and edges. Furthermore, to handle challenging scenarios that are beyond the limits of classical sensor based VO solutions, we investigate a recently emerging sensor — the event camera and study on event-based mapping — one of the key problems in the event-based VO/SLAM. The main achievements are summarized as follows. First, we revisit an old topic on relative pose estimation: accurately and robustly estimating the fundamental matrix given a collection of independently estimated homograhies. Three classical methods are reviewed and then we show a simple but nontrivial two-step normalization within the direct linear method that achieves similar performance to the less attractive and more computationally intensive hallucinated points based method. Second, an efficient 3D rotation estimation algorithm for depth cameras in piece-wise planar environments is presented. It shows that by using surface normal vectors as an input, planar modes in the corresponding density distribution function can be discovered and continuously tracked using efficient non-parametric estimation techniques. The relative rotation can be estimated by registering entire bundles of planar modes by using robust L1-norm minimization. Third, an efficient alternative to the iterative closest point algorithm for real-time tracking of modern depth cameras in ManhattanWorlds is developed. We exploit the common orthogonal structure of man-made environments in order to decouple the estimation of the rotation and the three degrees of freedom of the translation. The derived camera orientation is absolute and thus free of long-term drift, which in turn benefits the accuracy of the translation estimation as well. Fourth, we look into a more general structural regularity—edges. A real-time VO system that uses Canny edges is proposed for RGB-D cameras. Two novel alternatives to classical distance transforms are developed with great properties that significantly improve the classical Euclidean distance field based methods in terms of efficiency, accuracy and robustness. Finally, to deal with challenging scenarios that go beyond what standard RGB/RGB-D cameras can handle, we investigate the recently emerging event camera and focus on the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping

Design study for LANDSAT D attitude control system

A design and performance evaluation is presented for the LANDSAT D attitude control system (ACS). Control and configuration of the gimballed Ku-band antenna system for communication with the tracking and data relay satellite (TDRS). Control of the solar array drive considered part of the ACS is also addressed

Design study for LANDSAT-D attitude control system

The gimballed Ku-band antenna system for communication with TDRS was studied. By means of an error analysis it was demonstrated that the antenna cannot be open loop pointed to TDRS by an onboard programmer, but that an autotrack system was required. After some tradeoffs, a two-axis, azimuth-elevation type gimbal configuration was recommended for the antenna. It is shown that gimbal lock only occurs when LANDSAT-D is over water where a temporary loss of the communication link to TDRS is of no consequence. A preliminary gimbal control system design is also presented. A digital computer program was written that computes antenna gimbal angle profiles, assesses percent antenna beam interference with the solar array, and determines whether the spacecraft is over land or water, a lighted earth or a dark earth, and whether the spacecraft is in eclipse

Modeling and visualization of medical anesthesiology acts

Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaIn recent years, medical visualization has evolved from simple 2D images on a light board to 3D computarized images. This move enabled doctors to find better ways of planning surgery and to diagnose patients. Although there is a great variety of 3D medical imaging software, it falls short when dealing with anesthesiology acts. Very little anaesthesia related work has been done. As a consequence, doctors and medical students have had little support to study the subject of anesthesia in the human body. We all are aware of how costly can be setting medical experiments, covering not just medical aspects but ethical and financial ones as well. With this work we hope to contribute for having better medical visualization tools in the area of anesthesiology. Doctors and in particular medical students should study anesthesiology acts more efficiently. They should be able to identify better locations to administrate the anesthesia, to study how long does it take for the anesthesia to affect patients, to relate the effect on patients with quantity of anaesthesia provided, etc. In this work, we present a medical visualization prototype with three main functionalities: image pre-processing, segmentation and rendering. The image pre-processing is mainly used to remove noise from images, which were obtained via imaging scanners. In the segmentation stage it is possible to identify relevant anatomical structures using proper segmentation algorithms. As a proof of concept, we focus our attention in the lumbosacral region of the human body, with data acquired via MRI scanners. The segmentation we provide relies mostly in two algorithms: region growing and level sets. The outcome of the segmentation implies the creation of a 3D model of the anatomical structure under analysis. As for the rendering, the 3D models are visualized using the marching cubes algorithm. The software we have developed also supports time-dependent data. Hence, we could represent the anesthesia flowing in the human body. Unfortunately, we were not able to obtain such type of data for testing. But we have used human lung data to validate this functionality

Real-space imaging of polar and elastic nano-textures in thin films via inversion of diffraction data

Exploiting the emerging nanoscale periodicities in epitaxial, single-crystal thin films is an exciting direction in quantum materials science: confinement and periodic distortions induce novel properties. The structural motifs of interest are ferroelastic, ferroelectric, multiferroic, and, more recently, topologically protected magnetization and polarization textures. A critical step towards heterostructure engineering is understanding their nanoscale structure, best achieved through real-space imaging. X-ray Bragg coherent diffractive imaging visualizes sub-picometer crystalline displacements with tens of nanometers spatial resolution. Yet, it is limited to objects spatially confined in all three dimensions and requires highly coherent, laser-like x-rays. Here we lift the confinement restriction by developing real-space imaging of periodic lattice distortions: we combine an iterative phase retrieval algorithm with unsupervised machine learning to invert the diffuse scattering in conventional x-ray reciprocal-space mapping into real-space images of polar and elastic textures in thin epitaxial films. We first demonstrate our imaging in PbTiO3/SrTiO3 superlattices to be consistent with published phase-field model calculations. We then visualize strain-induced ferroelastic domains emerging during the metal-insulator transition in Ca2RuO4 thin films. Instead of homogeneously transforming into a low-temperature structure (like in bulk), the strained Mott insulator splits into nanodomains with alternating lattice constants, as confirmed by cryogenic scanning transmission electron microscopy. Our study reveals the type, size, orientation, and crystal displacement field of the nano-textures. The non-destructive imaging of textures promises to improve models for their dynamics and enable advances in quantum materials and microelectronics

Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard

This paper presents a novel method for fully automatic and convenient extrinsic calibration of a 3D LiDAR and a panoramic camera with a normally printed chessboard. The proposed method is based on the 3D corner estimation of the chessboard from the sparse point cloud generated by one frame scan of the LiDAR. To estimate the corners, we formulate a full-scale model of the chessboard and fit it to the segmented 3D points of the chessboard. The model is fitted by optimizing the cost function under constraints of correlation between the reflectance intensity of laser and the color of the chessboard's patterns. Powell's method is introduced for resolving the discontinuity problem in optimization. The corners of the fitted model are considered as the 3D corners of the chessboard. Once the corners of the chessboard in the 3D point cloud are estimated, the extrinsic calibration of the two sensors is converted to a 3D-2D matching problem. The corresponding 3D-2D points are used to calculate the absolute pose of the two sensors with Unified Perspective-n-Point (UPnP). Further, the calculated parameters are regarded as initial values and are refined using the Levenberg-Marquardt method. The performance of the proposed corner detection method from the 3D point cloud is evaluated using simulations. The results of experiments, conducted on a Velodyne HDL-32e LiDAR and a Ladybug3 camera under the proposed re-projection error metric, qualitatively and quantitatively demonstrate the accuracy and stability of the final extrinsic calibration parameters.Comment: 20 pages, submitted to the journal of Remote Sensin

SURVEILLANCE MISSION PLANNING FOR UAVS IN GPS-DENIED URBAN ENVIRONMENT

Ph.DDOCTOR OF PHILOSOPH