Visual Odometry and Sparse Scene Reconstruction for UAVs with a Multi-Fisheye Camera System

Autonomously operating UAVs demand a fast localization for navigation, to actively explore unknown areas and to create maps. For pose estimation, many UAV systems make use of a combination of GPS receivers and inertial sensor units (IMU). However, GPS signal coverage may go down occasionally, especially in the close vicinity of objects, and precise IMUs are too heavy to be carried by lightweight UAVs. This and the high cost of high quality IMU motivate the use of inexpensive vision based sensors for localization using visual odometry or visual SLAM (simultaneous localization and mapping) techniques. The first contribution of this thesis is a more general approach to bundle adjustment with an extended version of the projective coplanarity equation which enables us to make use of omnidirectional multi-camera systems which may consist of fisheye cameras that can capture a large field of view with one shot. We use ray directions as observations instead of image points which is why our approach does not rely on a specific projection model assuming a central projection. In addition, our approach allows the integration and estimation of points at infinity, which classical bundle adjustments are not capable of. We show that the integration of far or infinitely far points stabilizes the estimation of the rotation angles of the camera poses. In its second contribution, we employ this approach to bundle adjustment in a highly integrated system for incremental pose estimation and mapping on light-weight UAVs. Based on the image sequences of a multi-camera system our system makes use of tracked feature points to incrementally build a sparse map and incrementally refines this map using the iSAM2 algorithm. Our system is able to optionally integrate GPS information on the level of carrier phase observations even in underconstrained situations, e.g. if only two satellites are visible, for georeferenced pose estimation. This way, we are able to use all available information in underconstrained GPS situations to keep the mapped 3D model accurate and georeferenced. In its third contribution, we present an approach for re-using existing methods for dense stereo matching with fisheye cameras, which has the advantage that highly optimized existing methods can be applied as a black-box without modifications even with cameras that have field of view of more than 180 deg. We provide a detailed accuracy analysis of the obtained dense stereo results. The accuracy analysis shows the growing uncertainty of observed image points of fisheye cameras due to increasing blur towards the image border. Core of the contribution is a rigorous variance component estimation which allows to estimate the variance of the observed disparities at an image point as a function of the distance of that point to the principal point. We show that this improved stochastic model provides a more realistic prediction of the uncertainty of the triangulated 3D points.Autonom operierende UAVs benötigen eine schnelle Lokalisierung zur Navigation, zur Exploration unbekannter Umgebungen und zur Kartierung. Zur Posenbestimmung verwenden viele UAV-Systeme eine Kombination aus GPS-Empfängern und Inertial-Messeinheiten (IMU). Die Verfügbarkeit von GPS-Signalen ist jedoch nicht überall gewährleistet, insbesondere in der Nähe abschattender Objekte, und präzise IMUs sind für leichtgewichtige UAVs zu schwer. Auch die hohen Kosten qualitativ hochwertiger IMUs motivieren den Einsatz von kostengünstigen bildgebenden Sensoren zur Lokalisierung mittels visueller Odometrie oder SLAM-Techniken zur simultanen Lokalisierung und Kartierung. Im ersten wissenschaftlichen Beitrag dieser Arbeit entwickeln wir einen allgemeineren Ansatz für die Bündelausgleichung mit einem erweiterten Modell für die projektive Kollinearitätsgleichung, sodass auch omnidirektionale Multikamerasysteme verwendet werden können, welche beispielsweise bestehend aus Fisheyekameras mit einer Aufnahme einen großen Sichtbereich abdecken. Durch die Integration von Strahlrichtungen als Beobachtungen ist unser Ansatz nicht von einem kameraspezifischen Abbildungsmodell abhängig solange dieses der Zentralprojektion folgt. Zudem erlaubt unser Ansatz die Integration und Schätzung von unendlich fernen Punkten, was bei klassischen Bündelausgleichungen nicht möglich ist. Wir zeigen, dass durch die Integration weit entfernter und unendlich ferner Punkte die Schätzung der Rotationswinkel der Kameraposen stabilisiert werden kann. Im zweiten Beitrag verwenden wir diesen entwickelten Ansatz zur Bündelausgleichung für ein System zur inkrementellen Posenschätzung und dünnbesetzten Kartierung auf einem leichtgewichtigen UAV. Basierend auf den Bildsequenzen eines Mulitkamerasystems baut unser System mittels verfolgter markanter Bildpunkte inkrementell eine dünnbesetzte Karte auf und verfeinert diese inkrementell mittels des iSAM2-Algorithmus. Unser System ist in der Lage optional auch GPS Informationen auf dem Level von GPS-Trägerphasen zu integrieren, wodurch sogar in unterbestimmten Situation - beispielsweise bei nur zwei verfügbaren Satelliten - diese Informationen zur georeferenzierten Posenschätzung verwendet werden können. Im dritten Beitrag stellen wir einen Ansatz zur Verwendung existierender Methoden für dichtes Stereomatching mit Fisheyekameras vor, sodass hoch optimierte existierende Methoden als Black Box ohne Modifzierungen sogar mit Kameras mit einem Gesichtsfeld von mehr als 180 Grad verwendet werden können. Wir stellen eine detaillierte Genauigkeitsanalyse basierend auf dem Ergebnis des dichten Stereomatchings dar. Die Genauigkeitsanalyse zeigt, wie stark die Genauigkeit beobachteter Bildpunkte bei Fisheyekameras zum Bildrand aufgrund von zunehmender Unschärfe abnimmt. Das Kernstück dieses Beitrags ist eine Varianzkomponentenschätzung, welche die Schätzung der Varianz der beobachteten Disparitäten an einem Bildpunkt als Funktion von der Distanz dieses Punktes zum Hauptpunkt des Bildes ermöglicht. Wir zeigen, dass dieses verbesserte stochastische Modell eine realistischere Prädiktion der Genauigkeiten der 3D Punkte ermöglicht

Global optimisation of multiple gravity assist trajectories

Multiple gravity assist (MGA) trajectories represent a particular class of space trajectories in which a spacecraft exploits the encounter with one or more celestial bodies to change its velocity vector; they have been essential to reach high Delta-v targets with low propellant consumption. The search for optimal transfer trajectories can be formulated as a mixed combinatorial-continuous global optimisation problem; however, it is known that the problem is difficult to solve, especially if deep space manoeuvres (DSM) are considered. This thesis addresses the automatic design of MGA trajectories through global search techniques, in answer to the requirements of having a large number of mission options in a short time, during the preliminary design phase. Two different approaches are presented. The first is a two-level approach: a number of feasible planetary sequences are initially generated; then, for each one, families of the MGA trajectories are built incrementally. The whole transfer is decomposed into sub-problems of smaller dimension and complexity, and the trajectory is progressively composed by solving one problem after the other. At each incremental step, a stochastic search identifies sets of feasible solutions: this region is preserved, while the rest of the search space is pruned out. The process iterates by adding one planet-to-planet leg at a time and pruning the unfeasible portion of the solution space. Therefore, when another leg is added to the trajectory, only the feasible set for the previous leg is considered and the search space is reduced. It is shown, through comparative tests, how the proposed incremental search performs an effective pruning of the search space, providing families of optimal solutions with a lower computational cost than a non-incremental approach. Known deterministic and stochastic methods are used for the comparison. The algorithm is applied to real MGA case studies, including the ESA missions BepiColombo and Laplace. The second approach performs an integrated search for the planetary sequence and the associated trajectories. The complete design of an MGA trajectory is formulated as an autonomous planning and scheduling problem. The resulting scheduled plan provides the planetary sequence for a MGA trajectory and a good estimation of the optimality of the associated trajectories. For each departure date, a full tree of possible transfers from departure to destination is generated. An algorithm inspired by Ant Colony Optimization (ACO) is devised to explore the space of possible plans. The ants explore the tree from departure to destination, adding one node at a time, using a probability function to select one of the feasible directions. Unlike standard ACO, a taboo-based heuristics prevents ants from re-exploring the same solutions. This approach is applied to the design of optimal transfers to Saturn (inspired by Cassini) and to Mercury, and it demonstrated to be very competitive against known traditional stochastic population-based techniques

Wavelets and wavelet-like transforms on the sphere and their application to geophysical data inversion

Many flexible parameterizations exist to represent data on the sphere. In addition to the venerable spherical harmonics, we have the Slepian basis, harmonic splines, wavelets and wavelet-like Slepian frames. In this paper we focus on the latter two: spherical wavelets developed for geophysical applications on the cubed sphere, and the Slepian "tree", a new construction that combines a quadratic concentration measure with wavelet-like multiresolution. We discuss the basic features of these mathematical tools, and illustrate their applicability in parameterizing large-scale global geophysical (inverse) problems.Comment: 15 pages, 11 figures, submitted to the Proceedings of the SPIE 2011 conference Wavelets and Sparsity XI

Near Zone: Basic scattering code user's manual with space station applications

The Electromagnetic Code - Basic Scattering Code, Version 3, is a user oriented computer code to analyze near and far zone patterns of antennas in the presence of scattering structures, to provide coupling between antennas in a complex environment, and to determine radiation hazard calculations at UHF and above. The analysis is based on uniform asymptotic techniques formulated in terms of the Uniform Geometrical Theory of Diffraction (UTD). Complicated structures can be simulated by arbitrarily oriented flat plates and an infinite ground plane that can be perfectly conducting or dielectric. Also, perfectly conducting finite elliptic cylinder, elliptic cone frustum sections, and finite composite ellipsoids can be used to model the superstructure of a ship, the body of a truck, and airplane, a satellite, etc. This manual gives special consideration to space station modeling applications. This is a user manual designed to give an overall view of the operation of the computer code, to instruct a user in how to model structures, and to show the validity of the code by comparing various computed results against measured and alternative calculations such as method of moments whenever available

HSC: A spectral clustering algorithm combined with hierarchical method

Most of the traditional clustering algorithms are poor for clustering more complex structures other than the convex spherical sample space. In the past few years, several spectral clustering algorithms were proposed to cluster arbitrarily shaped data in various real applications. However, spectral clustering relies on the dataset where each cluster is approximately well separated to a certain extent. In the case that the cluster has an obvious inflection point within a non-convex space, the spectral clustering algorithm would mistakenly recognize one cluster to be different clusters. In this paper, we propose a novel spectral clustering algorithm called HSC combined with hierarchical method, which obviates the disadvantage of the spectral clustering by not using the misleading information of the noisy neighboring data points. The simple clustering procedure is applied to eliminate the misleading information, and thus the HSC algorithm could cluster both convex shaped data and arbitrarily shaped data more efficiently and accurately. The experiments on both synthetic data sets and real data sets show that HSC outperforms other popular clustering algorithms. Furthermore, we observed that HSC can also be used for the estimation of the number of clusters. © CTU FTS 2013

Doctor of Philosophy

dissertationMagnetic resonance-guided focused ultrasound surgery (MRgFUS) is a noninvasive means of causing selective tissue necrosis using high-power ultrasound and MR temperature imaging. Inhomogeneities in the medium of propagation can cause significant distortion of the ultrasound beam, resulting in changes in focal-zone amplitude, location and shape. Current ultrasound beam simulation techniques are either only applicable to homogeneous media or are relatively slow in calculating power deposition patterns in inhomogeneous media. Further, these techniques use table-value estimates of the acoustic parameters for predicting ultrasound beam propagation in inhomogeneous media, resulting in at best an approximate power deposition pattern. This work improves numerical analysis of ultrasound beam propagation by developing techniques for: 1) fast, accurate predictions of ultrasound beam propagation in inhomogeneous media, 2) noninvasive estimation of acoustic parameters (speed of sound and attenuation coefficient) of tissue types present in inhomogeneous media, 3) noninvasive determination of changes in tissue acoustic properties due to treatment. These beam simulation techniques utilizing subject-specific tissue parameters will rapidly predict power deposition patterns in real patient geometries and estimate changes in tissue acoustic parameters during treatment, leading to treatment-responsive patientspecific treatment plans that will improve the safety, efficacy and effectiveness of MRgFUS

On 3-D inelastic analysis methods for hot section components (base program)

A 3-D Inelastic Analysis Method program is described. This program consists of a series of new computer codes embodying a progression of mathematical models (mechanics of materials, special finite element, boundary element) for streamlined analysis of: (1) combustor liners, (2) turbine blades, and (3) turbine vanes. These models address the effects of high temperatures and thermal/mechanical loadings on the local (stress/strain)and global (dynamics, buckling) structural behavior of the three selected components. Three computer codes, referred to as MOMM (Mechanics of Materials Model), MHOST (Marc-Hot Section Technology), and BEST (Boundary Element Stress Technology), have been developed and are briefly described in this report