Real-time Collision Risk Estimation based on Stochastic Reachability Spaces

International audienceEstimating the risk of collision with other road users is one of the most important modules to ensure safety in autonomous driving scenarios. In this paper, we propose new probabilistic models to obtain Stochastic Reachability Spaces for vehicles and pedestrians detected in the scene. We then exploit these probabilistic predictions of the road-users' future positions, along with the expected ego-vehicle trajectory, to estimate the probability of collision risk in real-time. The proposed stochastic models only depend on the velocity, acceleration, tracked bounding box, and the class of the detected object. This information can easily be obtained through off-the-shelf 3D object detection frameworks. As a result, the proposed approach for collision risk estimation is widely applicable to a variety of autonomous vehicle platforms. To validate our approach, initially we test the stochastic motion prediction on the KITTI dataset. Further experiments in the CARLA simulator, by reproducing realistic collision scenarios, have the goal of demonstrating the effectiveness of the collision risk assessment and are compared with an alternative approach

Nachweislich sichere Bewegungsplanung für autonome Fahrzeuge durch Echtzeitverifikation

This thesis introduces fail-safe motion planning as the first approach to guarantee legal safety of autonomous vehicles in arbitrary traffic situations. The proposed safety layer verifies whether intended trajectories comply with legal safety and provides fail-safe trajectories when intended trajectories result in safety-critical situations. The presented results indicate that the use of fail-safe motion planning can drastically reduce the number of traffic accidents.Die vorliegende Arbeit führt ein neuartiges Verifikationsverfahren ein, mit dessen Hilfe zum ersten Mal die verkehrsregelkonforme Sicherheit von autonomen Fahrzeugen gewährleistet werden kann. Das Verifikationsverfahren überprüft, ob geplante Trajektorien sicher sind und generiert Rückfalltrajektorien falls diese zu einer unsicheren Situation führen. Die Ergebnisse zeigen, dass die Verwendung des Verfahrens zu einer deutlichen Reduktion von Verkehrsunfällen führt

Modeling and Development of Human Interface for Pedestrian Simulator

According to Traveler opinion and perception survey of 2005, 107.4 million Americans use walking as regular mode of travel, which amounts to 51% of American population. In 2009, 4092 pedestrian fatalities have been reported nationwide with a fatality rate of 1.33 which totals 59,000 crashes. Also, pedestrians are over represented in crash data by accounting more than 12% of fatalities but on 10.9% of trips. This makes a perfect case for understanding the causes behind such statistics, calling for a continuous research on pedestrians walking behavior and their interactions with surroundings. Current research in pedestrian simulation focuses on surveys and mathematical simulation models such as macroscopic and microscopic dynamic models involves autonomous entities. The surveys represent the perception of individual while mathematical simulation severely limits the capacity to capture effect of human factors in the understanding of pedestrian interactions. Complicated psychological models are used to a certain extent for understanding of such problems but are incapable to estimate the diversity of human behavior. To capture tendencies of people, they need to be a part of research, under a safe and controlled environment. In this thesis, an attempt has been made to develop a module which can be used to track human walk gesture and map it to actual human walk. Then, this module could be implemented in a system aimed to understand pedestrian behavior. Following are the accomplishments of this thesis. * Built an API to use with software interface to capture human motion - Explored arduino based wearable interface to capture human motion. - Explored Kinect based video interface to capture human motion. - Defined gestures and identified configurations for least difficult setup and calibration process. - Wrote the software interface for a Kinect based system (video interface). * Built a mathematical framework for abstracted dynamical system, for the purpose of pedestrian interface in simulation engine. - Obtained mathematical model for human walk. - Obtained conversion to non-holonomical system for human walk. - Programmed the mathematical model into the API. Eventually this is expected to contribute towards state-of-the-art researches which aim at understanding pedestrian dynamics in transportation safety and planning. The module described is expected to work real-time as a separate entity

Clothoid-based Planning and Control in Intelligent Vehicles (Autonomous and Manual-Assisted Driving)

[EN] Nowadays, there are many electronic products that incorporate elements and features coming from the research in the field of mobile robotics. For instance, the well-known vacuum cleaning robot Roomba by iRobot, which belongs to the field of service robotics, one of the most active within the sector. There are also numerous autonomous robotic systems in industrial warehouses and plants. It is the case of Autonomous Guided Vehicles (AGVs), which are able to drive completely autonomously in very structured environments. Apart from industry and consumer electronics, within the automotive field there are some devices that give intelligence to the vehicle, derived in most cases from advances in mobile robotics. In fact, more and more often vehicles incorporate Advanced Driver Assistance Systems (ADAS), such as navigation control with automatic speed regulation, lane change and overtaking assistant, automatic parking or collision warning, among other features. However, despite all the advances there are some problems that remain unresolved and can be improved. Collisions and rollovers stand out among the most common accidents of vehicles with manual or autonomous driving. In fact, it is almost impossible to guarantee driving without accidents in unstructured environments where vehicles share the space with other moving agents, such as other vehicles and pedestrians. That is why searching for techniques to improve safety in intelligent vehicles, either autonomous or manual-assisted driving, is still a trending topic within the robotics community. This thesis focuses on the design of tools and techniques for planning and control of intelligent vehicles in order to improve safety and comfort. The dissertation is divided into two parts, the first one on autonomous driving and the second one on manual-assisted driving. The main link between them is the use of clothoids as mathematical formulation for both trajectory generation and collision detection. Among the problems solved the following stand out: obstacle avoidance, rollover avoidance and advanced driver assistance to avoid collisions with pedestrians.[ES] En la actualidad se comercializan infinidad de productos de electrónica de consumo que incorporan elementos y características procedentes de avances en el sector de la robótica móvil. Por ejemplo, el conocido robot aspirador Roomba de la empresa iRobot, el cual pertenece al campo de la robótica de servicio, uno de los más activos en el sector. También hay numerosos sistemas robóticos autónomos en almacenes y plantas industriales. Es el caso de los vehículos autoguiados (AGVs), capaces de conducir de forma totalmente autónoma en entornos muy estructurados. Además de en la industria y en electrónica de consumo, dentro del campo de la automoción también existen dispositivos que dotan de cierta inteligencia al vehículo, derivados la mayoría de las veces de avances en robótica móvil. De hecho, cada vez con mayor frecuencia los vehículos incorporan sistemas avanzados de asistencia al conductor (ADAS por sus siglas en inglés), tales como control de navegación con regulación automática de velocidad, asistente de cambio de carril y adelantamiento, aparcamiento automático o aviso de colisión, entre otras prestaciones. No obstante, pese a todos los avances siguen existiendo problemas sin resolver y que pueden mejorarse. La colisión y el vuelco destacan entre los accidentes más comunes en vehículos con conducción tanto manual como autónoma. De hecho, la dificultad de conducir en entornos desestructurados compartiendo el espacio con otros agentes móviles, tales como coches o personas, hace casi imposible garantizar la conducción sin accidentes. Es por ello que la búsqueda de técnicas para mejorar la seguridad en vehículos inteligentes, ya sean de conducción autónoma o manual asistida, es un tema que siempre está en auge en la comunidad robótica. La presente tesis se centra en el diseño de herramientas y técnicas de planificación y control de vehículos inteligentes, para la mejora de la seguridad y el confort. La disertación se ha dividido en dos partes, la primera sobre conducción autónoma y la segunda sobre conducción manual asistida. El principal nexo de unión es el uso de clotoides como elemento de generación de trayectorias y detección de colisiones. Entre los problemas que se resuelven destacan la evitación de obstáculos, la evitación de vuelcos y la asistencia avanzada al conductor para evitar colisiones con peatones.[CA] En l'actualitat es comercialitzen infinitat de productes d'electrònica de consum que incorporen elements i característiques procedents d'avanços en el sector de la robòtica mòbil. Per exemple, el conegut robot aspirador Roomba de l'empresa iRobot, el qual pertany al camp de la robòtica de servici, un dels més actius en el sector. També hi ha nombrosos sistemes robòtics autònoms en magatzems i plantes industrials. És el cas dels vehicles autoguiats (AGVs), els quals són capaços de conduir de forma totalment autònoma en entorns molt estructurats. A més de en la indústria i en l'electrònica de consum, dins el camp de l'automoció també existeixen dispositius que doten al vehicle de certa intel·ligència, la majoria de les vegades derivats d'avanços en robòtica mòbil. De fet, cada vegada amb més freqüència els vehicles incorporen sistemes avançats d'assistència al conductor (ADAS per les sigles en anglés), com ara control de navegació amb regulació automàtica de velocitat, assistent de canvi de carril i avançament, aparcament automàtic o avís de col·lisió, entre altres prestacions. No obstant això, malgrat tots els avanços segueixen existint problemes sense resoldre i que poden millorar-se. La col·lisió i la bolcada destaquen entre els accidents més comuns en vehicles amb conducció tant manual com autònoma. De fet, la dificultat de conduir en entorns desestructurats compartint l'espai amb altres agents mòbils, tals com cotxes o persones, fa quasi impossible garantitzar la conducció sense accidents. És per això que la recerca de tècniques per millorar la seguretat en vehicles intel·ligents, ja siguen de conducció autònoma o manual assistida, és un tema que sempre està en auge a la comunitat robòtica. La present tesi es centra en el disseny d'eines i tècniques de planificació i control de vehicles intel·ligents, per a la millora de la seguretat i el confort. La dissertació s'ha dividit en dues parts, la primera sobre conducció autònoma i la segona sobre conducció manual assistida. El principal nexe d'unió és l'ús de clotoides com a element de generació de trajectòries i detecció de col·lisions. Entre els problemes que es resolen destaquen l'evitació d'obstacles, l'evitació de bolcades i l'assistència avançada al conductor per evitar col·lisions amb vianants.Girbés Juan, V. (2016). Clothoid-based Planning and Control in Intelligent Vehicles (Autonomous and Manual-Assisted Driving) [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/65072TESI

Interactive Motion Prediction using Game Theory

This thesis considers the motion prediction problem in 2 locations where typically humans interact: a pedestrian hallway and a highway.\nThis study is obtained converting these scenarios in games, where each human is a player with a set of actions. Following the features of the game theory, we will make predictions on the motion of the players through the computation of the related Nash equilibria

Software architectural design for safety in Automated Parking System

The automotive industry has seen a revolution brought about by self-driving cars. However, one of the main challenges facing autonomous driving systems is ensuring safety in the absence of a supervising driver and verifying safe vehicle behaviour under various circumstances. Autonomous Driving Systems (ADS), due to their complexity, cannot be solved straightforwardly without proper structure. Thus, they need a well-defined architecture to guide their development with requirements that involve modularity, scalability, and maintainability among other properties. To help overcome some of the challenges, this master thesis defines and implements in a simulated environment an automated parking system that complies with industrial and safety standards. The work has been divided into four parts. Firstly, the safety rules for the development of an autonomous function have been analysed. Secondly, the use cases and system requirements have been defined following the needs of the automated parking system. Thirdly, the system has been implemented in the simulation environment with a structure based on a widely adopted automotive standard. The final result is the software architecture of an autonomous vehicle with automated parking functionality. This concept has been validated within the virtual environment together with the integration of the AUTOSAR runtime environment, which the communication between components and mode switching functionality in the CARLA simulation environment. The result of this project shows the benefit of integrating architecture and simulation, thus easing the development and testing of future autonomous systems

Game-Theoretic Safety Assurance for Human-Centered Robotic Systems

In order for autonomous systems like robots, drones, and self-driving cars to be reliably introduced into our society, they must have the ability to actively account for safety during their operation. While safety analysis has traditionally been conducted offline for controlled environments like cages on factory floors, the much higher complexity of open, human-populated spaces like our homes, cities, and roads makes it unviable to rely on common design-time assumptions, since these may be violated once the system is deployed. Instead, the next generation of robotic technologies will need to reason about safety online, constructing high-confidence assurances informed by ongoing observations of the environment and other agents, in spite of models of them being necessarily fallible.This dissertation aims to lay down the necessary foundations to enable autonomous systems to ensure their own safety in complex, changing, and uncertain environments, by explicitly reasoning about the gap between their models and the real world. It first introduces a suite of novel robust optimal control formulations and algorithmic tools that permit tractable safety analysis in time-varying, multi-agent systems, as well as safe real-time robotic navigation in partially unknown environments; these approaches are demonstrated on large-scale unmanned air traffic simulation and physical quadrotor platforms. After this, it draws on Bayesian machine learning methods to translate model-based guarantees into high-confidence assurances, monitoring the reliability of predictive models in light of changing evidence about the physical system and surrounding agents. This principle is first applied to a general safety framework allowing the use of learning-based control (e.g. reinforcement learning) for safety-critical robotic systems such as drones, and then combined with insights from cognitive science and dynamic game theory to enable safe human-centered navigation and interaction; these techniques are showcased on physical quadrotors—flying in unmodeled wind and among human pedestrians—and simulated highway driving. The dissertation ends with a discussion of challenges and opportunities ahead, including the bridging of safety analysis and reinforcement learning and the need to ``close the loop'' around learning and adaptation in order to deploy increasingly advanced autonomous systems with confidence

Contribution to the long term prediction of motion trajectories

Viele Anwendungen in der mobilen und kognitiven Robotik erfordern einen Prädiktionsmechanismus, um die zukünftigen Aufenthaltsorte bewegter Objekte zu schätzen. Ein autonomes Auto muss beispielsweise die Absichten der anderen Verkehrsteilnehmer schätzen können, um Kollisionen zu vermeiden und die Verkehrsregeln einzuhalten. Ein Serviceroboter muss hingegen in der Lage sein, die Bewegungsspuren der Personen in seiner Umgebung vorherzusagen, um in einer sozial akzeptablen Art und Weise zu navigieren und die Passanten nicht zu behindern. Fast alle Prädiktionsalgorithmen, die in der Literatur zu finden sind, beschäftigen sich mit der Kurzzeitprädiktion und sind auf spezielle Problemstellungen angepasst. Die Lösung einer neuen Problemstellung, welche eine Langzeitprädiktion benötigt (z.B. ein personalisierter Shopping-Assistent, oder eine intelligente Stauvorhersage), ist daher oft mit umfangreichem Forschungs- und Entwicklungsaufwand verbunden. Das Ziel dieser Dissertationsschrift liegt darin, sich dieses Defizits anzunehmen und der wissenschaftlichen Gemeinschaft ein vielseitig einsetzbares Langzeitprädiktionsframework zur Verfügung zu stellen. Das Framework trifft keine Annahmen über das jeweilige System und kann somit auf einfache Art und Weise an die spezifischen Anforderungen der individuellen Problemstellung angepasst werden. Das Framework selbst besteht aus drei Elementen: - Ein topologisches Modell, welches mit Hilfe eines Clustering Algorithmus anhand von Beobachtungen erstellt wird. Daraus resultiert ein topologischer Graph, welcher den Zustandsraum effizient abbildet und eine praktikable Repräsentation von Trajektorien ermöglicht. - Ein probabilistisches Modell, welches den topologischen Graphen um Übergangswahrscheinlichkeiten und Wahrscheinlichkeitsverteilungen der Übergangszeiten ergänzt. - Das eigentliche Prädiktionsframework, welches beide Modelle integriert. Mit Hilfe eines flussbasierten Algorithmus errechnet es für eine gegebene Eingabetrajektorie die zukünftigen Aufenthaltswahrscheinlichkeitsverteilungen über den gesamten Zustandsraum. Die im Rahmen dieser Arbeit durchgeführten Experimente zeigen, dass das vorgestellte Langzeitprädiktionsframework für Bewegungstrajektorien in der Lage ist, sich mit mehreren State of the Art Algorithmen zu messen, ohne dabei auf problemspezifische Bewegungsmodelle zurückzugreifen, physikalische Gesetze zu beachten, oder einschränkende Annahmen über den Zustandsraum des Systems zu treffen. Weiterhin enthalten die Experimente umfangreiche Auswertungen und Ergebnisse, um einen aussagekräftigen Vergleich mit künftigen Prädiktionsalgorithmen zu ermöglichen.Most applications of mobile and cognitive robotics require a prediction mechanism to estimate the future positions of moving objects. An autonomous car, for example, needs to determine the intentions of other traffic participants to avoid collisions and to obey the traffic rules. A service robot, on the other hand, needs to anticipate the paths of the surrounding pedestrians in order to move in a socially acceptable manner and to avoid awkward situations. Almost all prediction algorithms presented in literature mainly focus on the short term time horizon and usually give a solution tailored to a specific application. Thus, extensive research and development is necessary if new applications (e.g., a personalized shopping assistant or an intelligent traffic forecast) require a long term prediction mechanism. The goal of this thesis is to address this deficit and contribute a versatile long term prediction framework to the scientific community. It provides an algorithm which can easily be adapted to the individual task at hand by avoiding system specific assumptions such as motion characteristics, physical properties, or spatial restrictions. The framework consists of three elements: - A topological model which is based on observations and is created by utilizing a clustering algorithm. It incorporates a topological graph, sampling the state space efficiently and enabling a convenient representation of trajectories. -The topological model is enriched with a probabilistic model by encoding transitional probabilities and transitional time distributions into the graph. -Both models are integrated into the main prediction framework. By using a flow based algorithm, it provides the future probability distribution for a given input trajectory over the whole state space as a result. The experiments in this thesis show that the presented long term motion prediction framework is able to compete with a variety of state of the art algorithms. Furthermore, they include an extensive set of evaluations and results to enable an expressive comparison to future prediction algorithms

Abstractions, Analysis Techniques, and Synthesis of Scalable Control Strategies for Robot Swarms

Tasks that require parallelism, redundancy, and adaptation to dynamic, possibly hazardous environments can potentially be performed very efficiently and robustly by a swarm robotic system. Such a system would consist of hundreds or thousands of anonymous, resource-constrained robots that operate autonomously, with little to no direct human supervision. The massive parallelism of a swarm would allow it to perform effectively in the event of robot failures, and the simplicity of individual robots facilitates a low unit cost. Key challenges in the development of swarm robotic systems include the accurate prediction of swarm behavior and the design of robot controllers that can be proven to produce a desired macroscopic outcome. The controllers should be scalable, meaning that they ensure system operation regardless of the swarm size. This thesis presents a comprehensive approach to modeling a swarm robotic system, analyzing its performance, and synthesizing scalable control policies that cause the populations of different swarm elements to evolve in a specified way that obeys time and efficiency constraints. The control policies are decentralized, computed a priori, implementable on robots with limited sensing and communication capabilities, and have theoretical guarantees on performance. To facilitate this framework of abstraction and top-down controller synthesis, the swarm is designed to emulate a system of chemically reacting molecules. The majority of this work considers well-mixed systems when there are interaction-dependent task transitions, with some modeling and analysis extensions to spatially inhomogeneous systems. The methodology is applied to the design of a swarm task allocation approach that does not rely on inter-robot communication, a reconfigurable manufacturing system, and a cooperative transport strategy for groups of robots. The third application incorporates observations from a novel experimental study of the mechanics of cooperative retrieval in Aphaenogaster cockerelli ants. The correctness of the abstractions and the correspondence of the evolution of the controlled system to the target behavior are validated with computer simulations. The investigated applications form the building blocks for a versatile swarm system with integrated capabilities that have performance guarantees

Learning of Unknown Environments in Goal-Directed Guidance and Navigation Tasks: Autonomous Systems and Humans

University of Minnesota Ph.D. dissertation. December 2017. Major: Aerospace Engineering. Advisor: Berenice Mettler. 1 computer file (PDF); xvi, 176 pages.Guidance and navigation in unknown environments requires learning of the task environment simultaneous to path planning. Autonomous guidance in unknown environments requires a real-time integration of environment sensing, mapping, planning, trajectory generation, and tracking. For brute force optimal control, the spatial environment should be mapped accurately. The real-world environments are in general cluttered, complex, unknown, and uncertain. An accurate model of such environments requires to store an enormous amount of information and then that information has to be processed in optimal control formulation, which is not computationally cheap and efficient for online operations of autonomous guidance systems. On the contrary, humans and animals are in general able to navigate efficiently in unknown, complex, and cluttered environments. Like autonomous guidance systems, humans and animals also do not have unlimited information processing and sensing capacities due to their biological and physical constraints. Therefore, it is relevant to understand cognitive mechanisms that help humans learn and navigate efficiently in unknown environments. Such understanding can help to design planning algorithms that are computationally efficient as well as better understand how to improve human-machine interfaces in particular between operators and autonomous agents. This dissertation is organized in three parts: 1) computational investigation of environment learning in guidance and navigation (chapters 3 and 4), 2) investigation of human environment learning in guidance tasks (chapters 5 and 6), and 3) autonomous guidance framework based on a graph representation of environment using subgoals that are invariants in agent-environment interactions (chapter 7). In the first part, the dissertation presents a computational framework for learning autonomous guidance behavior in unknown or partially known environments. The learning framework uses a receding horizon trajectory optimization associated with a spatial value function (SVF). The SVF describes optimal (e.g. minimum time) guidance behavior represented as cost and velocity at any point in geographical space to reach a specified goal state. For guidance in unknown environments, a local SVF based on current vehicle state is updated online using environment data from onboard exteroceptive sensors. The proposed learning framework has the advantage in that it learns information directly relevant to the optimal guidance and control behavior enabling optimal trajectory planning in unknown or partially known environments. The learning framework is evaluated by measuring performance over successive runs in a 3-D indoor flight simulation. The test vehicle in the simulations is a Blade-Cx2 coaxial miniature helicopter. The environment is a priori unknown to the learning system. The dissertation investigates changes in performance, dynamic behavior, SVF, and control behavior in body frame, as a result of learning over successive runs. In the second part, the dissertation focuses on modeling and evaluating how a human operator learns an unknown task environment in goal-directed navigation tasks. Previous studies have showed that human pilots organize their guidance and perceptual behavior using the interaction patterns (IPs), i.e., invariants in their sensory-motor processes in interactions with the task space. However, previous studies were performed in known environments. In this dissertation, the concept of IPs is used to build a modeling and analysis framework to investigate human environment learning and decision-making in navigation of unknown environments. This approach emphasizes the agent dynamics (e.g., a vehicle controlled by a human operator), which is not typical in simultaneous navigation and environment learning studies. The framework is applied to analyze human data from simulated first-person guidance experiments in an obstacle field. Subjects were asked to perform multiple trials and find minimum-time routes between prespecified start and goal locations without priori knowledge of the environment. They used a joystick to control flight behavior and navigate in the environment. In the third part, the subgoal graph framework used to model and evaluate humans is extended to an autonomous guidance algorithm for navigation in unknown environments. The autonomous guidance framework based on subgoal graph is an improvement to the SVF based guidance and learning framework presented in the first part. The latter uses a grid representation of the environment, which is computationally costly in comparison to the graph based guidance model