Pedestrian and Vehicle Detection in Autonomous Vehicle Perception Systems—A Review

Autonomous Vehicles (AVs) have the potential to solve many traffic problems, such as accidents, congestion and pollution. However, there are still challenges to overcome, for instance, AVs need to accurately perceive their environment to safely navigate in busy urban scenarios. The aim of this paper is to review recent articles on computer vision techniques that can be used to build an AV perception system. AV perception systems need to accurately detect non-static objects and predict their behaviour, as well as to detect static objects and recognise the information they are providing. This paper, in particular, focuses on the computer vision techniques used to detect pedestrians and vehicles. There have been many papers and reviews on pedestrians and vehicles detection so far. However, most of the past papers only reviewed pedestrian or vehicle detection separately. This review aims to present an overview of the AV systems in general, and then review and investigate several detection computer vision techniques for pedestrians and vehicles. The review concludes that both traditional and Deep Learning (DL) techniques have been used for pedestrian and vehicle detection; however, DL techniques have shown the best results. Although good detection results have been achieved for pedestrians and vehicles, the current algorithms still struggle to detect small, occluded, and truncated objects. In addition, there is limited research on how to improve detection performance in difficult light and weather conditions. Most of the algorithms have been tested on well-recognised datasets such as Caltech and KITTI; however, these datasets have their own limitations. Therefore, this paper recommends that future works should be implemented on more new challenging datasets, such as PIE and BDD100K.EPSRC DTP PhD studentshi

Multimodal fusion architectures for pedestrian detection

Pedestrian detection provides a crucial functionality in many human-centric applications, such as video surveillance, urban scene analysis, and autonomous driving. Recently, multimodal pedestrian detection has received extensive attention since the fusion of complementary information captured by visible and infrared sensors enables robust human target detection under daytime and nighttime scenes. In this chapter, we systematically evaluate the performance of different multimodal fusion architectures in order to identify the optimal solutions for pedestrian detection. We made two important observations: (1) it is useful to combine the most commonly used concatenation fusion scheme with a global scene-aware mechanism to learn both human-related features and correlation between visible and thermal feature maps; (2) the two-stream segmentation supervision without multimodal fusion provides the most effective scheme to infuse segmentation information as supervision for learning human-related features. Based on these studies, we present a unified multimodal fusion framework for joint training of target detection and segmentation supervision which achieves the state-of-the-art multimodal pedestrian detection performance on the public KAIST benchmark dataset.</p

Pedestrian detection in far-infrared daytime images using a hierarchical codebook of SURF

One of the main challenges in intelligent vehicles concerns pedestrian detection for driving assistance. Recent experiments have showed that state-of-the-art descriptors provide better performances on the far-infrared (FIR) spectrum than on the visible one, even in daytime conditions, for pedestrian classification. In this paper, we propose a pedestrian detector with on-board FIR camera. Our main contribution is the exploitation of the specific characteristics of FIR images to design a fast, scale-invariant and robust pedestrian detector. Our system consists of three modules, each based on speeded-up robust feature (SURF) matching. The first module allows generating regions-of-interest (ROI), since in FIR images of the pedestrian shapes may vary in large scales, but heads appear usually as light regions. ROI are detected with a high recall rate with the hierarchical codebook of SURF features located in head regions. The second module consists of pedestrian full-body classification by using SVM. This module allows one to enhance the precision with low computational cost. In the third module, we combine the mean shift algorithm with inter-frame scale-invariant SURF feature tracking to enhance the robustness of our system. The experimental evaluation shows that our system outperforms, in the FIR domain, the state-of-the-art Haar-like Adaboost-cascade, histogram of oriented gradients (HOG)/linear SVM (linSVM) and MultiFtrpedestrian detectors, trained on the FIR images

Night time pedestrian detection for Advanced Driving Assistance Systems (ADAS) using near infrared images

From last decade, Safety plays a major role in automobile industry, which results in the invention of various safety measures such as air bags, central locking system, automatic breaking system, traffic signal detection etc. In such case pedestrian detection in night vision is one of the vital issues in advanced driving assistance systems. The main aim of the night vision systems is to avoid collision of vehicles with the pedestrians while driving on roads. It is very much important in night time, due to the varying light conditions it is very difficult to detect a pedestrian. With the presentation of night vision systems another sort of driver support is achieved, which can compensate the weaknesses of the human visual system after shutdown of sunlight. A NIR (Near Infrared) camera is used in this system to take images of a night scene. As there are large intra class variations in the pedestrian poses, a tree structured classifier is proposed here to handle the problem by training it with different subset of images and different sizes. This research work discusses about combination of Haar-Cascade and HOG-SVM (Histogram of Oriented Gradients-Support Vector Machine) for classification and validation. Haar-Cascade is trained such that to classify the full body of humans which eliminates most of the non-pedestrian regions. For refining the pedestrians after detection, a part based SVM classifier with HOG features is used. Upper and lower body part HOG features of the pedestrians are used for part based validation of detected bounding boxes. A full body validation scheme is also implemented using HOG-SVM when any one of the part based validation does not validate that particular part. Combination of the different types of complementary features yields better results. Experiments on test images determines that the proposed pedestrian detection system has a high detection rate and low false alarm rate since it works on part based validation process

Scenario-Driven Search for Pedestrians aimed at Triggering Non-Reversible Systems

Abstract-This paper presents the results of an innovative approach to pedestrian detection for automotive applications in which a non-reversible system is used; therefore the aim is to reach a very low false detection rate, ideally zero, by searching for pedestrians in specific areas only. The great advantages of such an approach are that pedestrian recognition is performed on limited image areas-therefore boosting its timing performance- and no assessment on the danger level is finally required before providing the result to either the driver or an on-board computer for automatic manoeuvres. This system has been extensively tested on two prototype vehicles equipped with one laserscanner, one camera, and brakeby-wire technology both in Italy and Korea; this paper describes the extensive tests and shows performance measurements. I

Fast pedestrian detection from a moving vehicle

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 69-71).This paper presents a method of real-time multi-modal pedestrian detection from a moving vehicle. The system uses both intensity and thermal images captured from cameras mounted at the front of the vehicle to train cascades of classifiers, which results in a detector that is able to detect a large percentage of pedestrians with very few false positives. The system has also been tested with inputs of high-resolution intensity images along with low-resolution thermal images, showing that the addition of even a low-resolution thermal camera may return better pedestrian detection results than using only intensity information alone.by Shuang You.M.Eng

Overview of Environment Perception for Intelligent Vehicles

This paper presents a comprehensive literature review on environment perception for intelligent vehicles. The state-of-the-art algorithms and modeling methods for intelligent vehicles are given, with a summary of their pros and cons. A special attention is paid to methods for lane and road detection, traffic sign recognition, vehicle tracking, behavior analysis, and scene understanding. In addition, we provide information about datasets, common performance analysis, and perspectives on future research directions in this area