Percepção do ambiente urbano e navegação usando visão robótica : concepção e implementação aplicado à veículo autônomo

Orientadores: Janito Vaqueiro Ferreira, Alessandro Corrêa VictorinoTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia MecânicaResumo: O desenvolvimento de veículos autônomos capazes de se locomover em ruas urbanas pode proporcionar importantes benefícios na redução de acidentes, no aumentando da qualidade de vida e também na redução de custos. Veículos inteligentes, por exemplo, frequentemente baseiam suas decisões em observações obtidas a partir de vários sensores tais como LIDAR, GPS e câmeras. Atualmente, sensores de câmera têm recebido grande atenção pelo motivo de que eles são de baixo custo, fáceis de utilizar e fornecem dados com rica informação. Ambientes urbanos representam um interessante mas também desafiador cenário neste contexto, onde o traçado das ruas podem ser muito complexos, a presença de objetos tais como árvores, bicicletas, veículos podem gerar observações parciais e também estas observações são muitas vezes ruidosas ou ainda perdidas devido a completas oclusões. Portanto, o processo de percepção por natureza precisa ser capaz de lidar com a incerteza no conhecimento do mundo em torno do veículo. Nesta tese, este problema de percepção é analisado para a condução nos ambientes urbanos associado com a capacidade de realizar um deslocamento seguro baseado no processo de tomada de decisão em navegação autônoma. Projeta-se um sistema de percepção que permita veículos robóticos a trafegar autonomamente nas ruas, sem a necessidade de adaptar a infraestrutura, sem o conhecimento prévio do ambiente e considerando a presença de objetos dinâmicos tais como veículos. Propõe-se um novo método baseado em aprendizado de máquina para extrair o contexto semântico usando um par de imagens estéreo, a qual é vinculada a uma grade de ocupação evidencial que modela as incertezas de um ambiente urbano desconhecido, aplicando a teoria de Dempster-Shafer. Para a tomada de decisão no planejamento do caminho, aplica-se a abordagem dos tentáculos virtuais para gerar possíveis caminhos a partir do centro de referencia do veículo e com base nisto, duas novas estratégias são propostas. Em primeiro, uma nova estratégia para escolher o caminho correto para melhor evitar obstáculos e seguir a tarefa local no contexto da navegação hibrida e, em segundo, um novo controle de malha fechada baseado na odometria visual e o tentáculo virtual é modelado para execução do seguimento de caminho. Finalmente, um completo sistema automotivo integrando os modelos de percepção, planejamento e controle são implementados e validados experimentalmente em condições reais usando um veículo autônomo experimental, onde os resultados mostram que a abordagem desenvolvida realiza com sucesso uma segura navegação local com base em sensores de câmeraAbstract: The development of autonomous vehicles capable of getting around on urban roads can provide important benefits in reducing accidents, in increasing life comfort and also in providing cost savings. Intelligent vehicles for example often base their decisions on observations obtained from various sensors such as LIDAR, GPS and Cameras. Actually, camera sensors have been receiving large attention due to they are cheap, easy to employ and provide rich data information. Inner-city environments represent an interesting but also very challenging scenario in this context, where the road layout may be very complex, the presence of objects such as trees, bicycles, cars might generate partial observations and also these observations are often noisy or even missing due to heavy occlusions. Thus, perception process by nature needs to be able to deal with uncertainties in the knowledge of the world around the car. While highway navigation and autonomous driving using a prior knowledge of the environment have been demonstrating successfully, understanding and navigating general inner-city scenarios with little prior knowledge remains an unsolved problem. In this thesis, this perception problem is analyzed for driving in the inner-city environments associated with the capacity to perform a safe displacement based on decision-making process in autonomous navigation. It is designed a perception system that allows robotic-cars to drive autonomously on roads, without the need to adapt the infrastructure, without requiring previous knowledge of the environment and considering the presence of dynamic objects such as cars. It is proposed a novel method based on machine learning to extract the semantic context using a pair of stereo images, which is merged in an evidential grid to model the uncertainties of an unknown urban environment, applying the Dempster-Shafer theory. To make decisions in path-planning, it is applied the virtual tentacle approach to generate possible paths starting from ego-referenced car and based on it, two news strategies are proposed. First one, a new strategy to select the correct path to better avoid obstacles and to follow the local task in the context of hybrid navigation, and second, a new closed loop control based on visual odometry and virtual tentacle is modeled to path-following execution. Finally, a complete automotive system integrating the perception, path-planning and control modules are implemented and experimentally validated in real situations using an experimental autonomous car, where the results show that the developed approach successfully performs a safe local navigation based on camera sensorsDoutoradoMecanica dos Sólidos e Projeto MecanicoDoutor em Engenharia Mecânic

An Evidential Filter for Indoor Navigation of a Mobile Robot in Dynamic Environment

International audienceRobots are destined to live with humans and perform tasks for them. In order to do that, an adapted representation of the world including human detection is required. Evidential grids enable the robot to handle partial information and ignorance, which can be useful in various situations. This paper deals with an audiovisual perception scheme of a robot in indoor environment (apartment, house..). As the robot moves, it must take into account its environment and the humans in presence. This article presents the key-stages of the multimodal fusion: an evidential grid is built from each modality using a modified Dempster combination, and a temporal fusion is made using an evidential filter based on an adapted version of the generalized bayesian theorem. This enables the robot to keep track of the state of its environment. A decision can then be made on the next move of the robot depending on the robot's mission and the extracted information. The system is tested on a simulated environment under realistic conditions

Cloud Update of Tiled Evidential Occupancy Grid Maps for the Multi-Vehicle Mapping

International audienceNowadays, many intelligent vehicles are equipped with various sensors to recognize their surrounding environment and to measure the motion or position of the vehicle. In addition, the number of intelligent vehicles equipped with a mobile Internet modem is increasing. Based on the sensors and Internet connection, the intelligent vehicles are able to share the sensor information with other vehicles via a cloud service. The sensor information sharing via the cloud service promises to improve the safe and efficient operation of the multiple intelligent vehicles. This paper presents a cloud update framework of occupancy grid maps for multiple intelligent vehicles in a large-scale environment. An evidential theory is applied to create the occupancy grid maps to address sensor disturbance such as measurement noise, occlusion and dynamic objects. Multiple vehicles equipped with LiDARs, motion sensors, and a low-cost GPS receiver create the evidential occupancy grid map (EOGM) for their passing trajectory based on GraphSLAM. A geodetic quad-tree tile system is applied to manage the EOGM, which provides a common tiling format to cover the large-scale environment. The created EOGM tiles are uploaded to EOGM cloud and merged with old EOGM tiles in the cloud using Dempster combination of evidential theory. Experiments were performed to evaluate the multiple EOGM mapping and the cloud update framework for large-scale road environment

Active Perception for Autonomous Systems : In a Deep Space Navigation Scenario

Autonomous systems typically pursue certain goals for an extended amount of time in a self-sustainable fashion. To this end, they are equipped with a set of sensors and actuators to perceive certain aspects of the world and thereupon manipulate it in accordance with some given goals. This kind of interaction can be thought of as a closed loop in which a perceive-reason-act process takes place. The bi-directional interface between an autonomous system and the outer world is then given by a sequence of imperfect observations of the world and corresponding controls which are as well imperfectly actuated. To be able to reason in such a setting, it is customary for an autonomous system to maintain a probabilistic state estimate. The quality of the estimate -- or its uncertainty -- is, in turn, dependent on the information acquired within the perceive-reason-act loop described above. Hence, this thesis strives to investigate the question of how to actively steer such a process in order to maximize the quality of the state estimate. The question will be approached by introducing different probabilistic state estimation schemes jointly working on a manifold-based encapsuled state representation. On top of the resultant state estimate different active perception approaches are introduced, which determine optimal actions with respect to uncertainty minimization. The informational value of the particular actions is given by the expected impact of measurements on the uncertainty. The latter can be obtained by different direct and indirect measures, which will be introduced and discussed. The active perception schemes for autonomous systems will be investigated with a focus on two specific deep space navigation scenarios deduced from a potential mining mission to the main asteroid belt. In the first scenario, active perception strategies are proposed, which foster the correctional value of the sensor information acquired within a heliocentric navigation approach. Here, the expected impact of measurements is directly estimated, thus omitting counterfactual updates of the state based on hypothetical actions. Numerical evaluations of this scenario show that active perception is beneficial, i.e., the quality of the state estimate is increased. In addition, it is shown that the more uncertain a state estimate is, the more the value of active perception increases. In the second scenario, active autonomous deep space navigation in the vicinity of asteroids is investigated. A trajectory and a map are jointly estimated by a Graph SLAM algorithm based on measurements of a 3D Flash-LiDAR. The active perception strategy seeks to trade-off the exploration of the asteroid against the localization performance. To this end, trajectories are generated as well as evaluated in a novel twofold approach specifically tailored to the scenario. Finally, the position uncertainty can be extracted from the graph structure and subsequently be used to dynamically control the trade-off between localization and exploration. In a numerical evaluation, it is shown that the localization performance of the Graph SLAM approach to navigation in the vicinity of asteroids is generally high. Furthermore, the active perception strategy is able to trade-off between localization performance and the degree of exploration of the asteroid. Finally, when the latter process is dynamically controlled, based on the current localization uncertainty, a joint improvement of localization as well as exploration performance can be achieved. In addition, this thesis comprises an excursion into active sensorimotor object recognition. A sensorimotor feature is derived from biological principles of the human perceptual system. This feature is then employed in different probabilistic classification schemes. Furthermore, it enables the implementation of an active perception strategy, which can be thought of as a feature selection process in a classification scheme. It is shown that those strategies might be driven by top-down factors, i.e., based on previously learned information, or by bottom-up factors, i.e., based on saliency detected in the currently considered data. Evaluations are conducted based on real data acquired by a camera mounted on a robotic arm as well as on datasets. It is shown that the integrated representation of perception and action fosters classification performance and that the application of an active perception strategy accelerates the classification process

Advances in the Bayesian Occupancy Filter framework using robust motion detection technique for dynamic environment monitoring

International audienceThe Bayesian Occupancy Filter provides a framework for grid-based monitoring of the dynamic environment. It allows to estimate dynamic grids, containing both information of occupancy and velocity. Clustering such grids then provides detection of the objects in the observed scene. In this paper we present recent improvements in this framework. First, multiple layers from a laser scanner are fused using opinion pool, to deal with conflicting information. Then a fast motion detection technique based on laser data and odometer/IMU information is used to separate the dynamic environment from the static one. This technique instead of performing a complete SLAM (Simultaneous Localization and Mapping) solution, is based on transferring occupancy information between consecutive data grids, the objective is to avoid false positives (static objects) like other DATMO approaches. Finally, we show the integration with Bayesian Occupancy Filter (BOF) and with the subsequent tracking module called Fast Clustering-Tracking Algorithm (FCTA). We especially show the improvements achieved in tracking results after this integration, for an intelligent vehicle application

Robust Dense Mapping for Large-Scale Dynamic Environments

We present a stereo-based dense mapping algorithm for large-scale dynamic urban environments. In contrast to other existing methods, we simultaneously reconstruct the static background, the moving objects, and the potentially moving but currently stationary objects separately, which is desirable for high-level mobile robotic tasks such as path planning in crowded environments. We use both instance-aware semantic segmentation and sparse scene flow to classify objects as either background, moving, or potentially moving, thereby ensuring that the system is able to model objects with the potential to transition from static to dynamic, such as parked cars. Given camera poses estimated from visual odometry, both the background and the (potentially) moving objects are reconstructed separately by fusing the depth maps computed from the stereo input. In addition to visual odometry, sparse scene flow is also used to estimate the 3D motions of the detected moving objects, in order to reconstruct them accurately. A map pruning technique is further developed to improve reconstruction accuracy and reduce memory consumption, leading to increased scalability. We evaluate our system thoroughly on the well-known KITTI dataset. Our system is capable of running on a PC at approximately 2.5Hz, with the primary bottleneck being the instance-aware semantic segmentation, which is a limitation we hope to address in future work. The source code is available from the project website (http://andreibarsan.github.io/dynslam).Comment: Presented at IEEE International Conference on Robotics and Automation (ICRA), 201

Development of a ground robot for indoor SLAM using Low‐Cost LiDAR and remote LabVIEW HMI

The simultaneous localization and mapping problem (SLAM) is crucial to autonomous navigation and robot mapping. The main purpose of this thesis is to develop a ground robot that implements SLAM to test the performance of the low‐cost RPLiDAR A1M8 by DFRobot. The HectorSLAM package, available in ROS was used with a Raspberry Pi to implement SLAM and build maps. These maps are sent to a remote desktop via TCP/IP communication to be displayed on a LabVIEW HMI where the user can also control robot. The LabVIEW HMI and the project in its entirety is intended to be as easy to use as possible to the layman, with many processes being automated to make this possible. The quality of the maps created by HectorSLAM and the RPLiDAR were evaluated both qualitatively and quanitatively to determine how useful the low‐cost LiDAR can be for this application. It is hoped that the apparatus developed in this project will be used with drones in the future for 3D mapping

Simultaneous Localization and Mapping (SLAM) for Autonomous Driving: Concept and Analysis

The Simultaneous Localization and Mapping (SLAM) technique has achieved astonishing progress over the last few decades and has generated considerable interest in the autonomous driving community. With its conceptual roots in navigation and mapping, SLAM outperforms some traditional positioning and localization techniques since it can support more reliable and robust localization, planning, and controlling to meet some key criteria for autonomous driving. In this study the authors first give an overview of the different SLAM implementation approaches and then discuss the applications of SLAM for autonomous driving with respect to different driving scenarios, vehicle system components and the characteristics of the SLAM approaches. The authors then discuss some challenging issues and current solutions when applying SLAM for autonomous driving. Some quantitative quality analysis means to evaluate the characteristics and performance of SLAM systems and to monitor the risk in SLAM estimation are reviewed. In addition, this study describes a real-world road test to demonstrate a multi-sensor-based modernized SLAM procedure for autonomous driving. The numerical results show that a high-precision 3D point cloud map can be generated by the SLAM procedure with the integration of Lidar and GNSS/INS. Online four–five cm accuracy localization solution can be achieved based on this pre-generated map and online Lidar scan matching with a tightly fused inertial system