Same Features, Different Day: Weakly Supervised Feature Learning for Seasonal Invariance

"Like night and day" is a commonly used expression to imply that two things are completely different. Unfortunately, this tends to be the case for current visual feature representations of the same scene across varying seasons or times of day. The aim of this paper is to provide a dense feature representation that can be used to perform localization, sparse matching or image retrieval, regardless of the current seasonal or temporal appearance. Recently, there have been several proposed methodologies for deep learning dense feature representations. These methods make use of ground truth pixel-wise correspondences between pairs of images and focus on the spatial properties of the features. As such, they don't address temporal or seasonal variation. Furthermore, obtaining the required pixel-wise correspondence data to train in cross-seasonal environments is highly complex in most scenarios. We propose Deja-Vu, a weakly supervised approach to learning season invariant features that does not require pixel-wise ground truth data. The proposed system only requires coarse labels indicating if two images correspond to the same location or not. From these labels, the network is trained to produce "similar" dense feature maps for corresponding locations despite environmental changes. Code will be made available at: https://github.com/jspenmar/DejaVu_Feature

The Revisiting Problem in Simultaneous Localization and Mapping: A Survey on Visual Loop Closure Detection

Where am I? This is one of the most critical questions that any intelligent system should answer to decide whether it navigates to a previously visited area. This problem has long been acknowledged for its challenging nature in simultaneous localization and mapping (SLAM), wherein the robot needs to correctly associate the incoming sensory data to the database allowing consistent map generation. The significant advances in computer vision achieved over the last 20 years, the increased computational power, and the growing demand for long-term exploration contributed to efficiently performing such a complex task with inexpensive perception sensors. In this article, visual loop closure detection, which formulates a solution based solely on appearance input data, is surveyed. We start by briefly introducing place recognition and SLAM concepts in robotics. Then, we describe a loop closure detection system's structure, covering an extensive collection of topics, including the feature extraction, the environment representation, the decision-making step, and the evaluation process. We conclude by discussing open and new research challenges, particularly concerning the robustness in dynamic environments, the computational complexity, and scalability in long-term operations. The article aims to serve as a tutorial and a position paper for newcomers to visual loop closure detection.Comment: 25 pages, 15 figure

Deep Learning Based Place Recognition for Challenging Environments

Visual based place recognition involves recognising familiar locations despite changes in environment or view-point of the camera(s) at the locations. There are existing methods that deal with these seasonal changes or view-point changes separately, but few methods exist that deal with these kind of changes simultaneously. Such robust place recognition systems are essential to long term localization and autonomy. Such systems should be able to deal both with conditional and viewpoint changes simultaneously. In recent times Convolutional Neural Networks (CNNs) have shown to outperform other state-of-the art method in task related to classi cation and recognition including place recognition. In this thesis, we present a deep learning based planar omni-directional place recognition approach that can deal with conditional and viewpoint variations together. The proposed method is able to deal with large viewpoint changes, where current methods fail. We evaluate the proposed method on two real world datasets dealing with four di erent seasons through out the year along with illumination changes and changes occurred in the environment across a period of 1 year respectively. We provide both quantitative (recall at 100% precision) and qualitative (confusion matrices) comparison of the basic pipeline for place recognition for the omni-directional approach with single-view and side-view camera approaches. The proposed approach is also shown to work very well across di erent seasons. The results prove the e cacy of the proposed method over the single-view and side-view cameras in dealing with conditional and large viewpoint changes in di erent conditions including illumination, weather, structural changes etc