End-to-End Learning of Semantic Grid Estimation Deep Neural Network with Occupancy Grids

International audienceWe propose semantic grid, a spatial 2D map of the environment around an autonomous vehicle consisting of cells which represent the semantic information of the corresponding region such as car, road, vegetation, bikes, etc. It consists of an integration of an occupancy grid, which computes the grid states with a Bayesian filter approach, and semantic segmentation information from monocular RGB images, which is obtained with a deep neural network. The network fuses the information and can be trained in an end-to-end manner. The output of the neural network is refined with a conditional random field. The proposed method is tested in various datasets (KITTI dataset, Inria-Chroma dataset and SYNTHIA) and different deep neural network architectures are compared

Impact of AI on Autonomous Driving

International audienceMotion Autonomy and Safety issues in Autonomous Vehicles are strongly dependent upon the capabilities and performances of Embedded Perception and Decision-making systems. This talk presents how it is possible to address these important issues using both Bayesian Perception and Machine Learning approaches. The talk will be illustrated using results obtained by Inria Grenoble Rhône-Alpes (France) in the scope of several R&D projects with IRT Nanoelec (French Technological Research Institute) and with several industrial companies such as Toyota or Renault

Leveraging Dynamic Occupancy Grids for 3D Object Detection in Point Clouds

International audienceTraditionally, point cloud-based 3D object detectors are trained on annotated, non-sequential samples taken from driving sequences (e.g. the KITTI dataset). However, by doing this, the developed algorithms renounce to exploit any dynamic information from the driving sequences. It is reasonable to think that this information, which is available at test time when deploying the models in the experimental vehicles, could have significant predictive potential for the object detection task. To study the advantages that this kind of information could provide, we construct a dataset of dynamic occupancy grid maps from the raw KITTI dataset and find the correspondence to each of the KITTI 3D object detection dataset samples. By training a Lidar-based state-of-the-art 3D object detector with and without the dynamic information we get insights into the predictive value of the dynamics. Our results show that having access to the environment dynamics improves by 27% the ability of the detection algorithm to predict the orientation of smaller obstacles such as pedestrians. Furthermore, the 3D and bird's eye view bounding box predictions for pedestrians in challenging cases also see a 7% improvement. Qualitatively speaking, the dynamics help with the detection of partially occluded and far-away obstacles. We illustrate this fact with numerous qualitative prediction results

Recognize Moving Objects Around an Autonomous Vehicle Considering a Deep-learning Detector Model and Dynamic Bayesian Occupancy

International audiencePerception systems on autonomous vehicles have the challenge of understanding the traffic scene in different situations. The fusion of redundant information obtained from different sources has been shown considerable progress under different methodologies to achieve this objective. However, new opportunities are available to obtain better fusion results with the advance of deep-learning models and computing hardware. In this paper, we aim to recognize moving objects in traffic scenes through the fusion of semantic information with occupancy-grid estimations. Our approach considers a deep-learning model with inference times between 22 to 55 milliseconds. Moreover, we use a Bayesian occupancy framework with a Highly-parallelized design to obtain the occupancygrid estimations.We validate our approach using experimental results with real-world data on urban scenery

Semantic Grid Estimation with Occupancy Grids and Semantic Segmentation Networks

International audienceWe propose a method to estimate the semantic grid for an autonomous vehicle. The semantic grid is a 2D bird's eye view map where the grid cells contain semantic characteristics such as road, car, pedestrian, signage, etc. We obtain the semantic grid by fusing the semantic segmentation information and an occupancy grid computed by using a Bayesian filter technique. To compute the semantic information from a monocular RGB image, we integrate segmentation deep neural networks into our model. We use a deep neural network to learn the relation between the semantic information and the occupancy grid which can be trained end-to-end extending our previous work on semantic grids. Furthermore, we investigate the effect of using a conditional random field to refine the results. Finally, we test our method on two datasets and compare different architecture types for semantic segmentation. We perform the experiments on KITTI dataset and Inria-Chroma dataset

Combining Bayesian and AI approaches for Autonomous Driving

International audienceInvited Keynote Talk. This talk addresses the exciting new concept of Autonomous Driving, as well as the technical questions and solutions associated with it. Emphasis will be placed on the scientific and technological challenges associated with issues of embedded perception, understanding of complex dynamic scenes and real-time driving decision-making. It will be shown how these problems can be tackled using Bayesian Perception, Artificial Intelligence and Machine Learning approaches. The talk will be illustrated using results obtained by Inria Grenoble Rhône-Alpes (France) in the scope of several R&D projects conducted in collaboration with IRT Nanoelec (French Technological Research Institute) and with several industrial companies such as Toyota or Renault

Embedded Bayesian Perception and Collision Risk Assessment (invited talk)

International audienceThe lack of robustness and of efficiency of current Embedded Perception and Decision-making systems is one of the major obstacles to a full deployment of self-driving cars. In this talk, it will be argued that three enabling technologies are required for improving the capabilities of such systems: (1) a framework for fusing multiple-sensor data in the presence of uncertainty and for interpreting in real-time the surrounding dynamic environment, (2) a method for predicting future environmental changes using perception history, contextual information and some prior knowledge, and (3) a decision-making approach having the capability to continuously evaluate the risk of future collisions and to provide on-line maneuvers recommendations for a safe navigation.New approaches developed at Inria for Embedded Multi-sensors Perception, Situation Awareness, Collision risk assessment and on-line Decision-making for safe navigation, will be presented. It will be shown that Bayesian approaches are mandatory for developing such technologies and for obtaining the required robustness in the presence of uncertainty and of complex dynamic situations.The talk is illustrated by experimental results obtained in real traffic situations, in the scope of several collaborative projects involving Toyota, Renault, CEA-LETI, or the French IRT (Technological Research Institute) Nanoelec

Dynamic Scene Understanding and Upcoming Collision Prediction to improve Autonomous Driving Safety: A Bayesian Approach

International audienceThanks to the recent strong involvement of the Web Giants (GAFA) and of numerous international industrial companies and startups in the fields of car production, mobile robotics and mobility services, the concepts of Autonomous Vehicles and of Future Mobility Services is progressively becoming a reality connected to a huge expected market. More and more pre-products and innovative mobility services are both proposed and intensively tested in real world conditions. This is for instance the case with the commercially available Autopilot system of Tesla, or with the concept of Robot Taxi currently under testing in some US and Asian cities by companies such as Uber, Waymo or nuTonomy. Several millions of miles have been cover in the last decade by autonomous or semi-autonomous vehicles operating in real traffic environments, but at the expense of some benign or serious accidents due to insufficient safety conditions.The objective of this talk is to give a brief analysis of the state of the art in the field of Autonomous Vehicles, before focusing on one of the current brake on the deployment of such a technology: The lack of robustness and of efficiency of current Embedded Perception and Decision-making systems. After having presented some new technologies and trends for addressing these important issues, the emphasis will put on “Bayesian approaches” that are increasingly used to obtain the required robustness in presence of both real world uncertainty and complex dynamic scenes. It will be shown that the concept of “Dynamic Occupancy Grids” is extremely useful for addressing the abovementioned robustness and efficiency requirements. The approach will be illustrated using interesting experimental results obtained at Inria and IRT Nanoelec (a French Technological Research Institute) in the scope of several collaborative projects and technological transfers with Toyota, Renault and with some Industrial partners of IRT Nanoelec

Embedded Sensor Fusion and Perception for Autonomous Vehicle

International audienceInvited Talk.This talk presents a novel Embedded Perception System based on a robust and efficient Bayesian Sensor Fusion approach. The system provides in real time (1) the state of the dynamic environments of the vehicle (free space, static obstacles, dynamic obstacles along with their respective motion fields, and unknown areas), (2) the predicted upcoming changes of the dynamic environment and (3) the estimated short-term collision risks (about 3s ahead). This approach has been developed and patented by Inria and IRT (French Technological Research Institute) Nanoelec. In 2018, an exploitation license was sold to Toyota Motor Europe and to an industrial company working in the field of Autonomous Shuttles (confidential). The approach is illustrated by some recent results obtained in cooperation with Toyota, Renault and the French IRT Nanoelec