Deconvolutional networks for point-cloud vehicle detection and tracking in driving scenarios

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Vehicle detection and tracking is a core ingredient for developing autonomous driving applications in urban scenarios. Recent image-based Deep Learning (DL) techniques are obtaining breakthrough results in these perceptive tasks. However, DL research has not yet advanced much towards processing 3D point clouds from lidar range-finders. These sensors are very common in autonomous vehicles since, despite not providing as semantically rich information as images, their performance is more robust under harsh weather conditions than vision sensors. In this paper we present a full vehicle detection and tracking system that works with 3D lidar information only. Our detection step uses a Convolutional Neural Network (CNN) that receives as input a featured representation of the 3D information provided by a Velodyne HDL-64 sensor and returns a per-point classification of whether it belongs to a vehicle or not. The classified point cloud is then geometrically processed to generate observations for a multi-object tracking system implemented via a number of Multi-Hypothesis Extended Kalman Filters (MH-EKF) that estimate the position and velocity of the surrounding vehicles. The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach. Our lidar-based approach uses about a 4% of the data needed for an image-based detector with similarly competitive results.Peer ReviewedPostprint (author's final draft

Vehicle Detection from 3D Lidar Using Fully Convolutional Network

Abstract-Convolutional network techniques have recently achieved great success in vision based detection tasks. This paper introduces the recent development of our research on transplanting the fully convolutional network technique to the detection tasks on 3D range scan data. Specifically, the scenario is set as the vehicle detection task from the range data of Velodyne 64E lidar. We proposes to present the data in a 2D point map and use a single 2D end-to-end fully convolutional network to predict the objectness confidence and the bounding boxes simultaneously. By carefully design the bounding box encoding, it is able to predict full 3D bounding boxes even using a 2D convolutional network. Experiments on the KITTI dataset shows the state-ofthe-art performance of the proposed method

Stairs detection with odometry-aided traversal from a wearable RGB-D camera

Stairs are one of the most common structures present in human-made scenarios, but also one of the most dangerous for those with vision problems. In this work we propose a complete method to detect, locate and parametrise stairs with a wearable RGB-D camera. Our algorithm uses the depth data to determine if the horizontal planes in the scene are valid steps of a staircase judging their dimensions and relative positions. As a result we obtain a scaled model of the staircase with the spatial location and orientation with respect to the subject. The visual odometry is also estimated to continuously recover the current position and orientation of the user while moving. This enhances the system giving the ability to come back to previously detected features and providing location awareness of the user during the climb. Simultaneously, the detection of the staircase during the traversal is used to correct the drift of the visual odometry. A comparison of results of the stair detection with other state-of-the-art algorithms was performed using public dataset. Additional experiments have also been carried out, recording our own natural scenes with a chest-mounted RGB-D camera in indoor scenarios. The algorithm is robust enough to work in real-time and even under partial occlusions of the stair

Combining Perception and Knowledge for Service Robotics

As the deployment of robots is shifting away from the industrial settings towards public and private sectors, the robots will have to get equipped with enough knowl- edge that will let them perceive, comprehend and act skillfully in their new work- ing environments. Unlike having a large degree of controlled environment variables characteristic for e.g. assembly lines, the robots active in shopping stores, museums or households will have to perform open-ended tasks and thus react to unforeseen events, self-monitor their activities, detect failures, recover from them and also learn and continuously update their knowledge. In this thesis we present a set of tools and algorithms for acquisition, interpreta- tion and reasoning about the environment models which enable the robots to act flexibly and skillfully in the afore mentioned environments. In particular our contri- butions beyond the state-of-the-art cover following four topics: a) semantic object maps which are the symbolic representations of indoor environments that robot can query for information, b) two algorithms for interactive segmentation of objects of daily use which enable the robots to recognise and grasp objects more robustly, c) an image point feature-based system for large scale object recognition, and finally, d) a system that combines statistical and logical knowledge for household domains and is able to answer queries such as Which objects are currently missing on a breakfast table? . Common to all contributions is that they are all knowledge-enabled in that they either use robot knowledge bases or ground knowledge structures into the robot s internal structures such as perception streams. Further, in all four cases we exploit the tight interplay between the robot s perceptual, reasoning and action skills which we believe is the key enabler for robots to act in unstructured environments. Most of the theoretical contributions of this thesis have also been implemented on TUM-James and TUM-Rosie robots and demonstrated to the spectators by having them perform various household chores. With those demonstrations we thoroughly validated the properties of the developed systems and showed the impossibility of having such tasks implemented without a knowledge-enabled backbone

Segmentation and Unsupervised Part-based Discovery of Repetitive Objects

Abstract—In this paper, we present an unsupervised technique to segment and detect objects in indoor environments. The main idea of this work is to identify object instances whenever there is evidence for at least one other occurence of an object of the same kind. In contrast to former approaches, we do not assume any given segmentation of the data, but instead estimate the segmentation and the existence of object instances concurrently. We apply graph-based clustering in feature and in geometric space to presegmented input data. Each segment is treated as a potential object part, and the inter-dependence of object labels assigned to part clusters are modeled using a Conditional Random Field (CRF) named the “parts graph”. Another CRF is then applied to the scene graph to smooth the class labels using the distributions obtained from the parts graph. First results on indoor 3D laser range data are evaluated and presented. I