Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras

Cameras are the primary sensor in automated driving systems. They provide high information density and are optimal for detecting road infrastructure cues laid out for human vision. Surround-view camera systems typically comprise of four fisheye cameras with 190{\deg}+ field of view covering the entire 360{\deg} around the vehicle focused on near-field sensing. They are the principal sensors for low-speed, high accuracy, and close-range sensing applications, such as automated parking, traffic jam assistance, and low-speed emergency braking. In this work, we provide a detailed survey of such vision systems, setting up the survey in the context of an architecture that can be decomposed into four modular components namely Recognition, Reconstruction, Relocalization, and Reorganization. We jointly call this the 4R Architecture. We discuss how each component accomplishes a specific aspect and provide a positional argument that they can be synergized to form a complete perception system for low-speed automation. We support this argument by presenting results from previous works and by presenting architecture proposals for such a system. Qualitative results are presented in the video at https://youtu.be/ae8bCOF77uY.Comment: Accepted for publication at IEEE Transactions on Intelligent Transportation System

High-Level Interpretation of Urban Road Maps Fusing Deep Learning-Based Pixelwise Scene Segmentation and Digital Navigation Maps

This paper addresses the problem of high-level road modeling for urban environments. Current approaches are based on geometric models that fit well to the road shape for narrow roads. However, urban environments are more complex and those models are not suitable for inner city intersections or other urban situations. The approach presented in this paper generates a model based on the information provided by a digital navigation map and a vision-based sensing module. On the one hand, the digital map includes data about the road type (residential, highway, intersection, etc.), road shape, number of lanes, and other context information such as vegetation areas, parking slots, and railways. On the other hand, the sensing module provides a pixelwise segmentation of the road using a ResNet-101 CNN with random data augmentation, as well as other hand-crafted features such as curbs, road markings, and vegetation. The high-level interpretation module is designed to learn the best set of parameters of a function that maps all the available features to the actual parametric model of the urban road, using a weighted F-score as a cost function to be optimized. We show that the presented approach eases the maintenance of digital maps using crowd-sourcing, due to the small number of data to send, and adds important context information to traditional road detection systems

Model Predictive Control System Design of a passenger car for Valet Parking Scenario

A recent expansion of passenger cars’ automated functions has led to increasingly challenging design problems for the engineers. Among this the development of Automated Valet Parking is the latest addition. The system represents the next evolution of automated system giving the vehicle greater autonomy: the efforts of most automotive OEMs go towards achieving market deployment of such automated function. To this end the focus of each OEM is on taking part to this competitive endeavor and succeed by developing a proprietary solution with the support of hardware and software suppliers. Within this framework the present work aims at developing an effective control strategy for the considered scenarios. In order to reach this goal a Model Predictive Control approach is employed taking advantage of previous works within the automotive OEM in the automated driving field. The control algorithm is developed in a Simulink® simulation according to the requirements of the application and tested; results show the control strategy successfully drives the vehicle on the predefined path

Comprehensive Survey and Analysis of Techniques, Advancements, and Challenges in Video-Based Traffic Surveillance Systems

The challenges inherent in video surveillance are compounded by a several factors, like dynamic lighting conditions, the coordination of object matching, diverse environmental scenarios, the tracking of heterogeneous objects, and coping with fluctuations in object poses, occlusions, and motion blur. This research endeavor aims to undertake a rigorous and in-depth analysis of deep learning- oriented models utilized for object identification and tracking. Emphasizing the development of effective model design methodologies, this study intends to furnish a exhaustive and in-depth analysis of object tracking and identification models within the specific domain of video surveillance

FisheyeMultiNet: Real-time Multi-task Learning Architecture for Surround-view Automated Parking System.

Automated Parking is a low speed manoeuvring scenario which is quite unstructured and complex, requiring full 360° near-field sensing around the vehicle. In this paper, we discuss the design and implementation of an automated parking system from the perspective of camera based deep learning algorithms. We provide a holistic overview of an industrial system covering the embedded system, use cases and the deep learning architecture. We demonstrate a real-time multi-task deep learning network called FisheyeMultiNet, which detects all the necessary objects for parking on a low-power embedded system. FisheyeMultiNet runs at 15 fps for 4 cameras and it has three tasks namely object detection, semantic segmentation and soiling detection. To encourage further research, we release a partial dataset of 5,000 images containing semantic segmentation and bounding box detection ground truth via WoodScape project [Yogamani et al., 2019]

A Vision Based Top-View Transformation Model for a Vehicle Parking Assistant

This paper proposes the Top-View Transformation Model for image coordinate transformation, which involves transforming a perspective projection image into its corresponding bird's eye vision. A fitting parameters searching algorithm estimates the parameters that are used to transform the coordinates from the source image. Using this approach, it is not necessary to provide any interior and exterior orientation parameters of the camera. The designed car parking assistant system can be installed at the rear end of the car, providing the driver with a clearer image of the area behind the car. The processing time can be reduced by storing and using the transformation matrix estimated from the first image frame for a sequence of video images. The transformation matrix can be stored as the Matrix Mapping Table, and loaded into the embedded platform to perform the transformation. Experimental results show that the proposed approaches can provide a clearer and more accurate bird's eye view to the vehicle driver

Evaluation of ADAS with a supported-driver model for desired allocation of tasks between human and technology performance

Partly automated driving is relevant for solving mobility problems, but also causes concerns with respect to the driver‟s reliability in task performance. The supported driver model presented in this paper is therefore intended to answer the question, what type of support and in which circumstances, will enhance the driver‟s ability to control the vehicle. It became apparent that prerequisites for performing tasks differ per driving task‟s type and require different support. The possible support for each driving task‟s type, has been combined with support-types to reduce the error causations from each different performance level (i.e. knowledge-based, rule-based and skill-based performance). The allocation of support in relation to performance level and driving task‟s type resulted in a supported driver model and this model relates the requested circumstances to appropriate support types. Among three tested ADAS systems, semi-automated parking showed best allocation of support; converting the demanding parallel parking task into a rather routine-like operation