Using Bayesian Programming for Multisensor Multi-Target Tracking in Automative Applications

A prerequisite to the design of future Advanced Driver Assistance Systems for cars is a sensing system providing all the information required for high-level driving assistance tasks. Carsense is a European project whose purpose is to develop such a new sensing system. It will combine different sensors (laser, radar and video) and will rely on the fusion of the information coming from these sensors in order to achieve better accuracy, robustness and an increase of the information content. This paper demonstrates the interest of using probabilistic reasoning techniques to address this challenging multi-sensor data fusion problem. The approach used is called Bayesian Programming. It is a general approach based on an implementation of the Bayesian theory. It was introduced rst to design robot control programs but its scope of application is much broader and it can be used whenever one has to deal with problems involving uncertain or incomplete knowledge

Holistic Temporal Situation Interpretation for Traffic Participant Prediction

For a profound understanding of traffic situations including a prediction of traf- fic participants’ future motion, behaviors and routes it is crucial to incorporate all available environmental observations. The presence of sensor noise and depen- dency uncertainties, the variety of available sensor data, the complexity of large traffic scenes and the large number of different estimation tasks with diverging requirements require a general method that gives a robust foundation for the de- velopment of estimation applications. In this work, a general description language, called Object-Oriented Factor Graph Modeling Language (OOFGML), is proposed, that unifies formulation of esti- mation tasks from the application-oriented problem description via the choice of variable and probability distribution representation through to the inference method definition in implementation. The different language properties are dis- cussed theoretically using abstract examples. The derivation of explicit application examples is shown for the automated driv- ing domain. A domain-specific ontology is defined which forms the basis for four exemplary applications covering the broad spectrum of estimation tasks in this domain: Basic temporal filtering, ego vehicle localization using advanced interpretations of perceived objects, road layout perception utilizing inter-object dependencies and finally highly integrated route, behavior and motion estima- tion to predict traffic participant’s future actions. All applications are evaluated as proof of concept and provide an example of how their class of estimation tasks can be represented using the proposed language. The language serves as a com- mon basis and opens a new field for further research towards holistic solutions for automated driving

Interactive Prediction and Planning for Autonomous Driving: from Algorithms to Fundamental Aspects

Inevitably, autonomous vehicles need to interact with other road participants in a variety of highly complex or critical driving scenarios. It is still an extremely challenging task even for the forefront companies or institutes to enable autonomous vehicles to interactively predict the behavior of others, and plan safe and high-quality motions accordingly. The major obstacles are not just originated from prediction and planning algorithms with insufficient performances. Several fundamental problems in the fields of interactive prediction and planning still remain open, such as formulation, representation and evaluation of interactive prediction methods, motion dataset with densely interactive driving behavior, as well as interface of interactive prediction and planning algorithms. The aforementioned fundamental aspects of interactive prediction and planning are addressed in this dissertation along with various kinds of algorithms. First, generic environmental representation for various scenarios with topological decomposition is constructed, and a corresponding planning algorithm is designed by combining graph search and optimization. Hard constraints in optimization-based planners are also incorporated into the training loss of imitation learning so that the policy net can generate safe and feasible motions in highly constrained scenarios. Unified problem formulation and motion representation are designed for different paradigms of interactive predictors such as planning-based prediction (inverse reinforcement learning), as well as probabilistic graphical models (hidden Markov model) and deep neural networks (mixture density network), which are utilized for the prediction/planning interface design and prediction benchmark. A framework combing decision network and graph-search/optimization/sample-based planner is proposed to achieve a driving strategy which is defensive to potential violations of others, but not overly conservatively to threats of low probabilities. Such driving strategy is achieved via experiments based on the aforementioned interactive prediction and planning algorithms with proper interface designed. These predictors are also evaluated from closed loop perspective considering planning fatality when using the prediction results instead of pure data approximation metrics. Finally, INTERACTION (INTERnational, Adversarial and Cooperative moTION) dataset with highly interactive driving scenarios and behavior from international locations is constructed with interaction density metric defined to compare different datasets. The dataset has been utilized for various behavior-related research areas such as prediction, planning, imitation learning and behavior modeling, and is inspiring new research fields such as representation learning, interaction extraction and scenario generation

Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection

Effective fusion of complementary information captured by multi-modal sensors (visible and infrared cameras) enables robust pedestrian detection under various surveillance situations (e.g. daytime and nighttime). In this paper, we present a novel box-level segmentation supervised learning framework for accurate and real-time multispectral pedestrian detection by incorporating features extracted in visible and infrared channels. Specifically, our method takes pairs of aligned visible and infrared images with easily obtained bounding box annotations as input and estimates accurate prediction maps to highlight the existence of pedestrians. It offers two major advantages over the existing anchor box based multispectral detection methods. Firstly, it overcomes the hyperparameter setting problem occurred during the training phase of anchor box based detectors and can obtain more accurate detection results, especially for small and occluded pedestrian instances. Secondly, it is capable of generating accurate detection results using small-size input images, leading to improvement of computational efficiency for real-time autonomous driving applications. Experimental results on KAIST multispectral dataset show that our proposed method outperforms state-of-the-art approaches in terms of both accuracy and speed