Visual Place Recognition in Changing Environments

Localization is an essential capability of mobile robots and place recognition is an important component of localization. Only having precise localization, robots can reliably plan, navigate and understand the environment around them. The main task of visual place recognition algorithms is to recognize based on the visual input if the robot has seen previously a given place in the environment. Cameras are one of the popular sensors robots get information from. They are lightweight, affordable, and provide detailed descriptions of the environment in the form of images. Cameras are shown to be useful for the vast variety of emerging applications, from virtual and augmented reality applications to autonomous cars or even fleets of autonomous cars. All these applications need precise localization. Nowadays, the state-of-the-art methods are able to reliably estimate the position of the robots using image streams. One of the big challenges still is the ability to localize a camera given an image stream in the presence of drastic visual appearance changes in the environment. Visual appearance changes may be caused by a variety of different reasons, starting from camera-related factors, such as changes in exposure time, camera position-related factors, e.g. the scene is observed from a different position or viewing angle, occlusions, as well as factors that stem from natural sources, for example seasonal changes, different weather conditions, illumination changes, etc. These effects change the way the same place in the environments appears in the image and can lead to situations where it becomes hard even for humans to recognize the places. Also, the performance of the traditional visual localization approaches, such as FABMAP or DBow, decreases dramatically in the presence of strong visual appearance changes. The techniques presented in this thesis aim at improving visual place recognition capabilities for robotic systems in the presence of dramatic visual appearance changes. To reduce the effect of visual changes on image matching performance, we exploit sequences of images rather than individual images. This becomes possible as robotic systems collect data sequentially and not in random order. We formulate the visual place recognition problem under strong appearance changes as a problem of matching image sequences collected by a robotic system at different points in time. A key insight here is the fact that matching sequences reduces the ambiguities in the data associations. This allows us to establish image correspondences between different sequences and thus recognize if two images represent the same place in the environment. To perform a search for image correspondences, we construct a graph that encodes the potential matches between the sequences and at the same time preserves the sequentiality of the data. The shortest path through such a data association graph provides the valid image correspondences between the sequences. Robots operating reliably in an environment should be able to recognize a place in an online manner and not after having recorded all data beforehand. As opposed to collecting image sequences and then determining the associations between the sequences offline, a real-world system should be able to make a decision for every incoming image. In this thesis, we therefore propose an algorithm that is able to perform visual place recognition in changing environments in an online fashion between the query and the previously recorded reference sequences. Then, for every incoming query image, our algorithm checks if the robot is in the previously seen environment, i.e. there exists a matching image in the reference sequence, as well as if the current measurement is consistent with previously obtained query images. Additionally, to be able to recognize places in an online manner, a robot needs to recognize the fact that it has left the previously mapped area as well as relocalize when it re-enters environment covered by the reference sequence. Thus, we relax the assumption that the robot should always travel within the previously mapped area and propose an improved graph-based matching procedure that allows for visual place recognition in case of partially overlapping image sequences. To achieve a long-term autonomy, we further increase the robustness of our place recognition algorithm by incorporating information from multiple image sequences, collected along different overlapping and non-overlapping routes. This allows us to grow the coverage of the environment in terms of area as well as various scene appearances. The reference dataset then contains more images to match against and this increases the probability of finding a matching image, which can lead to improved localization. To be able to deploy a robot that performs localization in large scaled environments over extended periods of time, however, collecting a reference dataset may be a tedious, resource consuming and in some cases intractable task. Avoiding an explicit map collection stage fosters faster deployment of robotic systems in the real world since no map has to be collected beforehand. By using our visual place recognition approach the map collection stage can be skipped, as we are able to incorporate the information from a publicly available source, e.g., from Google Street View, into our framework due to its general formulation. This automatically enables us to perform place recognition on already existing publicly available data and thus avoid costly mapping phase. In this thesis, we additionally show how to organize the images from the publicly available source into the sequences to perform out-of-the-box visual place recognition without previously collecting the otherwise required reference image sequences at city scale. All approaches described in this thesis have been published in peer-reviewed conference papers and journal articles. In addition to that, most of the presented contributions have been released publicly as open source software

Proceedings - 32. Workshop Computational Intelligence: Berlin, 1. - 2. Dezember 2022

Dieser Tagungsband enthält die Beiträge des 32. Workshops „Computational Intelligence“ des Fachausschusses 5.14 der VDI/VDE-Gesellschaft für Mess- und Automatisierungstechnik (GMA) der vom 1.12. – 2.12.2022 in Berlin stattfand. Die Schwerpunkte sind Methoden, Anwendungen und Tools für - Fuzzy-Systeme - Deep Learning - Machine Learning sowie der Methodenvergleich anhand von industriellen und Benchmark-Problemen

Proceedings - 32. Workshop Computational Intelligence: Berlin, 1. - 2. Dezember 2022

This conference volume contains the contributions of the 32nd workshop "Computational Intelligence" of the Technical Committee 5.14 of the VDI/VDE Society for Measurement and Automation Technology (GMA) of 1.12. – 2.12.2022 in Berlin. The focus is on methods, applications and tools for

Human factors aspects of control room design: Guidelines and annotated bibliography

A human factors analysis of the workstation design for the Earth Radiation Budget Satellite mission operation room is discussed. The relevance of anthropometry, design rules, environmental design goals, and the social-psychological environment are discussed

Experiments in object tracking in image sequences

This thesis explores three object tracking algorithms for image sequences. These algorithms include the ensemble tracker, the EM-like mean-shift colour-histogram tracker, and the wandering-stable-lost scale-invariant feature transform (WSL-SIFT) tracker. The algorithms are radically different from one another. Despite their differences, they are evaluated on the same publicly available, moderately sized, research data sets which include 129 test cases in 13 different scenes. The results aid in fostering an understanding of their respective behaviours and in highlighting their flaws and failures. Lastly, an implementation setup is described that is suited to large-scale, grid computing, batch testing of these algorithms. Results clearly indicate that none of the evaluated trackers are suited to general purpose use. However, one may intelligently choose a tracker for a well-defined application by analysing the known scene characteristics

Inferenz von Kreuzungsinformationen aus Flottendaten

Künftige Fahrerassistenzsysteme und hochautomatisierte Fahrfunktionen basieren auf digitalen Karten. Um den hohen Anforderungen an die Korrektheit und Aktualität der enthaltenen Informationen gerecht zu werden, stellt die vorliegende Arbeit neuartige automatisierte Verfahren zur Extraktion aktueller Karteninformationen aus Flottendaten vor. Der Fokus liegt dabei auf der Inferenz von statischen Kreuzungsinformationen mittels maschinellen Lernens und statistischer Verfahren

Proceedings der 11. Internationalen Tagung Wirtschaftsinformatik (WI2013) - Band 1

The two volumes represent the proceedings of the 11th International Conference on Wirtschaftsinformatik WI2013 (Business Information Systems). They include 118 papers from ten research tracks, a general track and the Student Consortium. The selection of all submissions was subject to a double blind procedure with three reviews for each paper and an overall acceptance rate of 25 percent. The WI2013 was organized at the University of Leipzig between February 27th and March 1st, 2013 and followed the main themes Innovation, Integration and Individualization.:Track 1: Individualization and Consumerization Track 2: Integrated Systems in Manufacturing Industries Track 3: Integrated Systems in Service Industries Track 4: Innovations and Business Models Track 5: Information and Knowledge ManagementDie zweibändigen Tagungsbände zur 11. Internationalen Tagung Wirtschaftsinformatik (WI2013) enthalten 118 Forschungsbeiträge aus zehn thematischen Tracks der Wirtschaftsinformatik, einem General Track sowie einem Student Consortium. Die Selektion der Artikel erfolgte nach einem Double-Blind-Verfahren mit jeweils drei Gutachten und führte zu einer Annahmequote von 25%. Die WI2013 hat vom 27.02. - 01.03.2013 unter den Leitthemen Innovation, Integration und Individualisierung an der Universität Leipzig stattgefunden.:Track 1: Individualization and Consumerization Track 2: Integrated Systems in Manufacturing Industries Track 3: Integrated Systems in Service Industries Track 4: Innovations and Business Models Track 5: Information and Knowledge Managemen