Adaptive Perception, State Estimation, and Navigation Methods for Mobile Robots

In this cumulative habilitation, publications with focus on robotic perception, self-localization, tracking, navigation, and human-machine interfaces have been selected. While some of the publications present research on a PR2 household robot in the Robotics Learning Lab of the University of California Berkeley on vision and machine learning tasks, most of the publications present research results while working at the AutoNOMOS-Labs at Freie Universität Berlin, with focus on control, planning and object tracking for the autonomous vehicles "MadeInGermany" and "e-Instein"

Assessment of Different Technologies for Improving Visibility during Foggy Weather in Mining and Transportation Sectors

Generally during foggy weather in winter season, mining operation remains suspended for hours due to problem of visibility. Foggy weather also leads to accidents, loss of life and infrastructure damages in mining and transportation sectors. This paper discusses about the existing technologies for improving visibility in transportation sector and suitability assessment of these technologies in mines for uninterrupted mining operations in foggy weathe

Resolving Class Imbalance for LiDAR-based Object Detector by Dynamic Weight Average and Contextual Ground Truth Sampling

An autonomous driving system requires a 3D object detector, which must perceive all present road agents reliably to navigate an environment safely. However, real-world driving datasets often suffer from the problem of data imbalance, which causes difficulties in training a model that works well across all classes, resulting in an undesired imbalanced sub-optimal performance. In this work, we propose a method to address this data imbalance problem. Our method consists of two main components: (i) a LiDAR-based 3D object detector with per-class multiple detection heads where losses from each head are modified by dynamic weight average to be balanced. (ii) Contextual ground truth (GT) sampling, where we improve conventional GT sampling techniques by leveraging semantic information to augment point cloud with sampled ground truth GT objects. Our experiment with KITTI and nuScenes datasets confirms our proposed method's effectiveness in dealing with the data imbalance problem, producing better detection accuracy compared to existing approaches.Comment: 10 page

Computer Vision and Sensor Fusion for Autonomous Vehicles

Cars, particularly manually-driven cars, are one of the most commonly used modes of transportation today. However, millions of people are either killed or left with disabilities annually due to road traffic accidents caused by human error or sensor failures. Despite that, a lot of people seem reluctant to look into alternatives to manually driven vehicle transportation. This is understandable as driving cars has been the trustworthy mode of transportation for many years, and it is widely used in everyday life around the world. However, technological advances in the fields of machine learning and cyber-physical systems contributed to the emergence of nearly or fully autonomous vehicles, or driverless cars, as a true viable alternative for the current human-controlled driving mode. The technology still has a long way to go, mainly because the advances in vision and depth measurement sensors such as LIDARs can not achieve the levels of safety needed to make fully autonomous cars. Progress on this front is being made every day, and it seems inevitable that they will be readily available in the near future. Our team aims to further investigate the application of Computer Vision and sensor fusion to achieve independent self-driving without external guides. To accomplish this, we combine a depth camera with a LiDAR to provide better coverage of the surroundings and allow more accurate detection and thus accurate avoidance of obstacles. We are mounting the vision system on a model driverless car and using the vision data to guide the car control system. A computer vision algorithm will be run by the NVIDIA Jetson Nano to determine what course of action the car should take. The final prototype should be capable of driving at a reasonable speed without colliding with any objects and making decisions such as braking or turning when necessary

Corridor Navigation for Monocular Vision Mobile Robots

Monocular vision robots use a single camera to process information about its environment. By analyzing this scene, the robot can determine the best navigation direction. Many modern approaches to robot hallway navigation involve using a plethora of sensors to detect certain features in the environment. This can be laser range finders, inertial measurement units, motor encoders, and cameras. By combining all these sensors, there is unused data which could be useful for navigation. To draw back and develop a baseline approach, this thesis explores the reliability and capability of solely using a camera for navigation. The basic navigation structure begins by taking frames from the camera and breaking them down to find the most prominent lines. The location where these lines intersect determine the forward direction to drive the robot. To improve the accuracy of navigation, algorithm improvements and additional features from the camera frames are used. This includes line intersection weighting to reduce noise from extraneous lines, floor segmentation to improve rotational stability, and person detection

Towards Large-Scale Small Object Detection: Survey and Benchmarks

With the rise of deep convolutional neural networks, object detection has achieved prominent advances in past years. However, such prosperity could not camouflage the unsatisfactory situation of Small Object Detection (SOD), one of the notoriously challenging tasks in computer vision, owing to the poor visual appearance and noisy representation caused by the intrinsic structure of small targets. In addition, large-scale dataset for benchmarking small object detection methods remains a bottleneck. In this paper, we first conduct a thorough review of small object detection. Then, to catalyze the development of SOD, we construct two large-scale Small Object Detection dAtasets (SODA), SODA-D and SODA-A, which focus on the Driving and Aerial scenarios respectively. SODA-D includes 24828 high-quality traffic images and 278433 instances of nine categories. For SODA-A, we harvest 2513 high resolution aerial images and annotate 872069 instances over nine classes. The proposed datasets, as we know, are the first-ever attempt to large-scale benchmarks with a vast collection of exhaustively annotated instances tailored for multi-category SOD. Finally, we evaluate the performance of mainstream methods on SODA. We expect the released benchmarks could facilitate the development of SOD and spawn more breakthroughs in this field. Datasets and codes are available at: \url{https://shaunyuan22.github.io/SODA}

A comparison of near-infrared and visible imaging for surveillance applications

A computer vision approach is investigated which has low computational complexity and which compares near-infrared and visible image systems. The target application is a surveillance system for pedestrian and vehicular traffic. Near-infrared light has potential benefits including non-visible illumination requirements. Image-processing and intelligent classification algorithms for monitoring pedestrians are implemented in outdoor and indoor environments with frequent traffic. The image set collected consists of persons walking in the presence of foreground as well as background objects at different times during the day. Image sets with nonperson objects, e.g. bicycles and vehicles, are also considered. The complex, cluttered environments are highly variable, e.g. shadows and moving foliage. The system performance for near-infrared images is compared to that of traditional visible images. The approach consists of thresholding an image and creating a silhouette of new objects in the scene. Filtering is used to eliminate noise. Twenty-four features are calculated by MATLAB♭ code for each identified object. These features are analyzed for usefulness in object discrimination. Minimal combinations of features are proposed and explored for effective automated discrimination. Features were used to train and test a variety of classification architectures. The results show that the algorithm can effectively manipulate near-infrared images and that effective object classification is possible even in the presence of system noise and environmental clutter. The potential for automated surveillance based on near-infrared imaging and automated feature processing are discussed --Abstract, page iii