Visual Place Recognition for Autonomous Robots

Autonomous robotics has been the subject of great interest within the research community over the past few decades. Its applications are wide-spread, ranging from health-care to manufacturing, goods transportation to home deliveries, site-maintenance to construction, planetary explorations to rescue operations and many others, including but not limited to agriculture, defence, commerce, leisure and extreme environments. At the core of robot autonomy lies the problem of localisation, i.e, knowing where it is and within the robotics community, this problem is termed as place recognition. Place recognition using only visual input is termed as Visual Place Recognition (VPR) and refers to the ability of an autonomous system to recall a previously visited place using only visual input, under changing viewpoint, illumination and seasonal conditions, and given computational and storage constraints. This thesis is a collection of 4 inter-linked, mutually-relevant but branching-out topics within VPR: 1) What makes a place/image worthy for VPR?, 2) How to define a state-of-the-art in VPR?, 3) Do VPR techniques designed for ground-based platforms extend to aerial platforms? and 4) Can a handcrafted VPR technique outperform deep-learning-based VPR techniques? Each of these questions is a dedicated, peer-reviewed chapter in this thesis and the author attempts to answer these questions to the best of his abilities. The worthiness of a place essentially refers to the salience and distinctiveness of the content in the image of this place. This salience is modelled as a framework, namely memorable-maps, comprising of 3 conjoint criteria: a) Human-memorability of an image, 2) Staticity and 3) Information content. Because a large number of VPR techniques have been proposed over the past 10-15 years, and due to the variation of employed VPR datasets and metrics for evaluation, the correct state-of-the-art remains ambiguous. The author levels this playing field by deploying 10 contemporary techniques on a common platform and use the most challenging VPR datasets to provide a holistic performance comparison. This platform is then extended to aerial place recognition datasets to answer the 3rd question above. Finally, the author designs a novel, handcrafted, compute-efficient and training-free VPR technique that outperforms state-of-the-art VPR techniques on 5 different VPR datasets

Distributed scene reconstruction from multiple mobile platforms

Recent research on mobile robotics has produced new designs that provide house-hold robots with omnidirectional motion. The image sensor embedded in these devices motivates the application of 3D vision techniques on them for navigation and mapping purposes. In addition to this, distributed cheapsensing systems acting as unitary entity have recently been discovered as an efficient alternative to expensive mobile equipment. In this work we present an implementation of a visual reconstruction method, structure from motion (SfM), on a low-budget, omnidirectional mobile platform, and extend this method to distributed 3D scene reconstruction with several instances of such a platform. Our approach overcomes the challenges yielded by the plaform. The unprecedented levels of noise produced by the image compression typical of the platform is processed by our feature filtering methods, which ensure suitable feature matching populations for epipolar geometry estimation by means of a strict quality-based feature selection. The robust pose estimation algorithms implemented, along with a novel feature tracking system, enable our incremental SfM approach to novelly deal with ill-conditioned inter-image configurations provoked by the omnidirectional motion. The feature tracking system developed efficiently manages the feature scarcity produced by noise and outputs quality feature tracks, which allow robust 3D mapping of a given scene even if - due to noise - their length is shorter than what it is usually assumed for performing stable 3D reconstructions. The distributed reconstruction from multiple instances of SfM is attained by applying loop-closing techniques. Our multiple reconstruction system merges individual 3D structures and resolves the global scale problem with minimal overlaps, whereas in the literature 3D mapping is obtained by overlapping stretches of sequences. The performance of this system is demonstrated in the 2-session case. The management of noise, the stability against ill-configurations and the robustness of our SfM system is validated on a number of experiments and compared with state-of-the-art approaches. Possible future research areas are also discussed

Development of an adaptive navigation system for indoor mobile handling and manipulation platforms

A fundamental technology enabling the autonomous behavior of mobile robotics is navigation. It is a main prerequisite for mobile robotics to fulfill high-level tasks such as handling and manipulation, and is often identified as one of the key challenges in mobile robotics. The mapping and localization as the basis for navigation are intensively researched in the last few decades. However, there are still challenges or problems needed to be solved for online operating in large-scale environments or running on low-cost and energy-saving embedded systems. In this work, new developments and usages of Light Detection And Ranging (LiDAR) based Simultaneous Localization And Mapping (SLAM) algorithms are presented. A key component of LiDAR based SLAM algorithms, the scan matching algorithm, is explored. Different scan matching algorithms are systemically experimented with different LiDARs for indoor home-like environments for the first time. The influence of properties of LiDARs in scan matching algorithms is quantitatively analyzed. Improvements to Bayes filter based and graph optimization based SLAMs are presented. The Bayes filter based SLAMs mainly use the current sensor information to find the best estimation. A new efficient implementation of Rao-Blackwellized Particle Filter based SLAM is presented. It is based on a pre-computed lookup table and the parallelization of the particle updating. The new implementation runs efficiently on recent multi-core embedded systems that fulfill low cost and energy efficiency requirements. In contrast to Bayes filter based methods, graph optimization based SLAMs utilize all the sensor information and minimize the total error in the system. A new real-time graph building model and a robust integrated Graph SLAM solution are presented. The improvements include the definition of unique direction norms for points or lines extracted from scans, an efficient loop closure detection algorithm, and a parallel and adaptive implementation. The developed algorithm outperforms the state-of-the-art algorithms in processing time and robustness especially in large-scale environments using embedded systems instead of high-end computation devices. The results of the work can be used to improve the navigation system of indoor autonomous robots, like domestic environments and intra-logistics.Eine der grundlegenden Funktionen, welche die Autonomie in der mobilen Robotik ermöglicht, ist die Navigation. Sie ist eine wesentliche Voraussetzung dafür, dass mobile Roboter selbständig anspruchsvolle Aufgaben erfüllen können. Die Umsetzung der Navigation wird dabei oft als eine der wichtigsten Herausforderungen identifiziert. Die Kartenerstellung und Lokalisierung als Grundlage für die Navigation wurde in den letzten Jahrzehnten intensiv erforscht. Es existieren jedoch immer noch eine Reihe von Problemen, z.B. die Anwendung auf große Areale oder bei der Umsetzung auf kostengünstigen und energiesparenden Embedded-Systemen. Diese Arbeit stellt neue Ansätze und Lösungen im Bereich der LiDAR-basierten simultanen Positionsbestimmung und Kartenerstellung (SLAM) vor. Eine Schlüsselkomponente der LiDAR-basierten SLAM, die so genannten Scan-Matching-Algorithmen, wird näher untersucht. Verschiedene Scan-Matching-Algorithmen werden zum ersten Mal systematisch mit verschiedenen LiDARs für den Innenbereich getestet. Der Einfluss von LiDARs auf die Eigenschaften der Algorithmen wird quantitativ analysiert. Verbesserungen an Bayes-filterbasierten und graphoptimierten SLAMs werden in dieser Arbeit vorgestellt. Bayes-filterbasierte SLAMs verwenden hauptsächlich die aktuellen Sensorinformationen, um die beste Schätzung zu finden. Eine neue effiziente Implementierung des auf Partikel-Filter basierenden SLAM unter der Verwendung einer Lookup-Tabelle und der Parallelisierung wird vorgestellt. Die neue Implementierung kann effizient auf aktuellen Embedded-Systemen laufen. Im Gegensatz dazu verwenden Graph-SLAMs alle Sensorinformationen und minimieren den Gesamtfehler im System. Ein neues Echtzeitmodel für die Grafenerstellung und eine robuste integrierte SLAM-Lösung werden vorgestellt. Die Verbesserungen umfassen die Definition von eindeutigen Richtungsnormen für Scan, effiziente Algorithmen zur Erkennung von Loop Closures und eine parallele und adaptive Implementierung. Der entwickelte und auf eingebetteten Systemen eingesetzte Algorithmus übertrifft die aktuellen Algorithmen in Geschwindigkeit und Robustheit, insbesondere für große Areale. Die Ergebnisse der Arbeit können für die Verbesserung der Navigation von autonomen Robotern im Innenbereich, häuslichen Umfeld sowie der Intra-Logistik genutzt werden

A multisensor SLAM for dense maps of large scale environments under poor lighting conditions

This thesis describes the development and implementation of a multisensor large scale autonomous mapping system for surveying tasks in underground mines. The hazardous nature of the underground mining industry has resulted in a push towards autonomous solutions to the most dangerous operations, including surveying tasks. Many existing autonomous mapping techniques rely on approaches to the Simultaneous Localization and Mapping (SLAM) problem which are not suited to the extreme characteristics of active underground mining environments. Our proposed multisensor system has been designed from the outset to address the unique challenges associated with underground SLAM. The robustness, self-containment and portability of the system maximize the potential applications.The multisensor mapping solution proposed as a result of this work is based on a fusion of omnidirectional bearing-only vision-based localization and 3D laser point cloud registration. By combining these two SLAM techniques it is possible to achieve some of the advantages of both approaches – the real-time attributes of vision-based SLAM and the dense, high precision maps obtained through 3D lasers. The result is a viable autonomous mapping solution suitable for application in challenging underground mining environments.A further improvement to the robustness of the proposed multisensor SLAM system is a consequence of incorporating colour information into vision-based localization. Underground mining environments are often dominated by dynamic sources of illumination which can cause inconsistent feature motion during localization. Colour information is utilized to identify and remove features resulting from illumination artefacts and to improve the monochrome based feature matching between frames.Finally, the proposed multisensor mapping system is implemented and evaluated in both above ground and underground scenarios. The resulting large scale maps contained a maximum offset error of ±30mm for mapping tasks with lengths over 100m

Dynamics of DNA Breathing and Folding for Molecular Recognition and Computation

This thesis is centered on the development of the molecular beacon, as a new DNA probe for DNA genotyping, D N A computation and biophysical studies of DNA conformations. Molecular beacons are single-stranded DNA molecules that form a stem-and-loop structure. A fluorophore and a quencher are grafted at their two ends to report their conformations: when the molecular beacon is closed, fluorophore and quencher are held in close proximity and the fluorescence is quenched; when the molecular beacon is open, fluorophore and quencher are far apart, and the fluorescence is restored. Molecular beacons are ideal DNA probes coupling conformational switch with fluorescence signal turning-ON. We use molecular beacons to study the molecular recognition of single-stranded DNA (ssDNA) oligonucleotide. We present a thermodynamic diagram to show that structural constraints make the molecular beacon highly sensitive to the presence of mismatches in its target. We introduce a sequence sensitivity parameter to quantitatively compare different DNA probes, and propose an algorithm to optimally tune the probe\u27s structure for enhanced sequence discrimination. Logic gates (OR and AND gates) using molecular beacons are designed to carry most elementary molecular computations. The conformational changes associated with such computations can be used to concatenate many chemical reactions, and carry out complex molecular computations. Molecular beacons are also ideal probes to study DNA secondary structures and their fluctuations. We develop the fluorescence correlation spectroscopy (FCS) technique to monitor the dynamics of relaxation of DNA conformational fluctuations. We first measure the opening and closing timescales of DNA hairpin-loops. Activation barriers for opening and closing for different loop lengths and sequences are analyzed to better account for the stability of DNA secondary structures. A sequence dependent rigidity of ssDNA has been discovered, and analyzed in terms of base stacking. We then use F C S to study the dynamics of double-stranded DNA (dsDNA) breathing modes with synthetic DNA constructs. The analysis of the base pairing fluctuation dynamics, monitored by fluorescence, unravels lifetimes of breathing modes ranging from 1/us to 1ms. Long-range distortions of the d s DNA have been unraveled for purine-rich sequences, of relevance to the specificity of transcription initiation in prokaryotes

Information-theoretic environment modeling for mobile robot localization

To enhance robotic computational efficiency without degenerating accuracy, it is imperative to fit the right and exact amount of information in its simplest form to the investigated task. This thesis conforms to this reasoning in environment model building and robot localization. It puts forth an approach towards building maps and localizing a mobile robot efficiently with respect to unknown, unstructured and moderately dynamic environments. For this, the environment is modeled on an information-theoretic basis, more specifically in terms of its transmission property. Subsequently, the presented environment model, which does not specifically adhere to classical geometric modeling, succeeds in solving the environment disambiguation effectively. The proposed solution lays out a two-level hierarchical structure for localization. The structure makes use of extracted features, which are stored in two different resolutions in a single hybrid feature-map. This enables dual coarse-topological and fine-geometric localization modalities. The first level in the hierarchy describes the environment topologically, where a defined set of places is described by a probabilistic feature representation. A conditional entropy-based criterion is proposed to quantify the transinformation between the feature and the place domains. This criterion provides a double benefit of pruning the large dimensional feature space, and at the same time selecting the best discriminative features that overcome environment aliasing problems. Features with the highest transinformation are filtered and compressed to form a coarse resolution feature-map (codebook). Localization at this level is conducted through place matching. In the second level of the hierarchy, the map is viewed in high-resolution, as consisting of non-compressed entropy-processed features. These features are additionally tagged with their position information. Given the identified topological place provided by the first level, fine localization corresponding to the second level is executed using feature triangulation. To enhance the triangulation accuracy, redundant features are used and two metric evaluating criteria are employ-ed; one for dynamic features and mismatches detection, and another for feature selection. The proposed approach and methods have been tested in realistic indoor environments using a vision sensor and the Scale Invariant Feature Transform local feature extraction. Through experiments, it is demonstrated that an information-theoretic modeling approach is highly efficient in attaining combined accuracy and computational efficiency performances for localization. It has also been proven that the approach is capable of modeling environments with a high degree of unstructuredness, perceptual aliasing, and dynamic variations (illumination conditions; scene dynamics). The merit of employing this modeling type is that environment features are evaluated quantitatively, while at the same time qualitative conclusions are generated about feature selection and performance in a robot localization task. In this way, the accuracy of localization can be adapted in accordance with the available resources. The experimental results also show that the hybrid topological-metric map provides sufficient information to localize a mobile robot on two scales, independent of the robot motion model. The codebook exhibits fast and accurate topological localization at significant compression ratios. The hierarchical localization framework demonstrates robustness and optimized space and time complexities. This, in turn, provides scalability to large environments application and real-time employment adequacies

An effective scene recognition strategy for biomimetic robotic navigation

Master'sMASTER OF ENGINEERIN

Biological Nanowires: Integration of the silver(I) base pair into DNA with nanotechnological and synthetic biological applications

Modern computing and mobile device technologies are now based on semiconductor technology with nanoscale components, i.e., nanoelectronics, and are used in an increasing variety of consumer, scientific, and space-based applications. This rise to global prevalence has been accompanied by a similarly precipitous rise in fabrication cost, toxicity, and technicality; and the vast majority of modern nanotechnology cannot be repaired in whole or in part. In combination with looming scaling limits, it is clear that there is a critical need for fabrication technologies that rely upon clean, inexpensive, and portable means; and the ideal nanoelectronics manufacturing facility would harness micro- and nanoscale fabrication and self-assembly techniques. The field of molecular electronics has promised for the past two decades to fill fundamental gaps in modern, silicon-based, micro- and nanoelectronics; yet molecular electronic devices, in turn, have suffered from problems of size, dispersion and reproducibility. In parallel, advances in DNA nanotechnology over the past several decades have allowed for the design and assembly of nanoscale architectures with single-molecule precision, and indeed have been used as a basis for heteromaterial scaffolds, mechanically-active delivery mechanisms, and network assembly. The field has, however, suffered for lack of meaningful modularity in function: few designs to date interact with their surroundings in more than a mechanical manner. As a material, DNA offers the promise of nanometer resolution, self-assembly, linear shape, and connectivity into branched architectures; while its biological origin offers information storage, enzyme-compatibility and the promise of biologically-inspired fabrication through synthetic biological means. Recent advances in DNA chemistry have isolated and characterized an orthogonal DNA base pair using standard nucleobases: by bridging the gap between mismatched cytosine nucleotides, silver(I) ions can be selectively incorporated into the DNA helix with atomic resolution. The goal of this thesis is to explore how this approach to “metallize” DNA can be combined with structural DNA nanotechnology as a step toward creating electronically-functional DNA networks. This work begins with a survey of applications for such a transformative technology, including nanoelectronic component fabrication for low-resource and space-based applications. We then investigate the assembly of linear Ag+-functionalized DNA species using biochemical and structural analyses to gain an understanding of the kinetics, yield, morphology, and behavior of this orthogonal DNA base pair. After establishing a protocol for high yield assembly in the presence of varying Ag+ functionalization, we investigate these linear DNA species using electrical means. First a method of coupling orthogonal DNA to single-walled carbon nanotubes (SWCNTs) is explored for self-assembly into nanopatterned transistor devices. Then we carry out scanning tunneling microscope (STM) break junction experiments on short polycytosine, polycationic DNA duplexes and find increased molecular conductance of at least an order of magnitude relative to the most conductive DNA analog. With an understanding of linear species from both a biochemical and nanoelectronic perspective, we investigate the assembly of nonlinear Ag+-functionalized DNA species. Using rational design principles gathered from the analysis of linear species, a de novo mathematical framework for understanding generalized DNA networks is developed. This provides the basis for a computational model built in Matlab that is able to design DNA networks and nanostructures using arbitrary base parity. In this way, DNA nanostructures are able to be designed using the dC:Ag+:dC base pair, as well as any similar nucleobase or DNA-inspired system (dT:Hg2+:dT, rA:rU, G4, XNA, LNA, PNA, etc.). With this foundation, three general classes of DNA tiles are designed with embedded nanowire elements: single crossover Holliday junction (HJ) tiles, T-junction (TJ) units, and double crossover (DX) tile pairs and structures. A library of orthogonal chemistry DNA nanotechnology is described, and future applications to nanomaterials and circuit architectures are discussed

Towards Robust Visual Localization in Challenging Conditions

Visual localization is a fundamental problem in computer vision, with a multitude of applications in robotics, augmented reality and structure-from-motion. The basic problem is to, based on one or more images, figure out the position and orientation of the camera which captured these images relative to some model of the environment. Current visual localization approaches typically work well when the images to be localized are captured under similar conditions compared to those captured during mapping. However, when the environment exhibits large changes in visual appearance, due to e.g. variations in weather, seasons, day-night or viewpoint, the traditional pipelines break down. The reason is that the local image features used are based on low-level pixel-intensity information, which is not invariant to these transformations: when the environment changes, this will cause a different set of keypoints to be detected, and their descriptors will be different, making the long-term visual localization problem a challenging one. In this thesis, four papers are included, which present work towards solving the problem of long-term visual localization. Three of the articles present ideas for how semantic information may be included to aid in the localization process: one approach relies only on the semantic information for visual localization, another shows how the semantics can be used to detect outlier feature correspondences, while the third presents a sequential localization algorithm which relies on the consistency of the reprojection of a semantic model, instead of traditional features. The final article is a benchmark paper, where we present three new benchmark datasets aimed at evaluating localization algorithms in the context of long-term visual localization