Learning Behavior Models for Interpreting and Predicting Traffic Situations

In this thesis, we present Bayesian state estimation and machine learning methods for predicting traffic situations. The cognitive ability to assess situations and behaviors of traffic participants, and to anticipate possible developments is an essential requirement for several applications in the traffic domain, especially for self-driving cars. We present a method for learning behavior models from unlabeled traffic observations and develop improved learning methods for decision trees

Belief State Planning for Autonomous Driving: Planning with Interaction, Uncertain Prediction and Uncertain Perception

This work presents a behavior planning algorithm for automated driving in urban environments with an uncertain and dynamic nature. The algorithm allows to consider the prediction uncertainty (e.g. different intentions), perception uncertainty (e.g. occlusions) as well as the uncertain interactive behavior of the other agents explicitly. Simulating the most likely future scenarios allows to find an optimal policy online that enables non-conservative planning under uncertainty

Path Planning and Performance Evaluation Strategies for Marine Robotic Systems

The field of marine robotics offers many new capabilities for completing dangerous missions such as deep-sea exploration and underwater demining. The harshness of marine environments, however, means that without effective onboard decision-making, vehicle loss or mission failure are likely. Thus, to enable more autonomous operation while building trust that these systems will perform as expected, this thesis develops improved path planning and testing strategies for two different types of marine robotic platforms. The first portion of the research focuses on improved environmental data collection with an autonomous underwater vehicle (AUV). Gaussian process-based modeling is combined with informative path planning to explore an environment, while preferentially collecting data in regions of interest that exhibit extreme sensory measurements. The performance of this adaptive data sampling framework with a torpedo-style AUV is studied in both simulation and field experiments. Results show that the proposed methodology is able to be fielded on an operational platform and collect measurements in regions of interest without sacrificing overall model fidelity of the full sampling area. The second portion of the research then focuses on autonomous surface vessel (ASV) navigation that must comply with international collision avoidance standards and basic ship handling principles. The approach introduces a novel quantification of good seamanship that is used within an ASV path planner to minimize the collision risk with other vessels. This approach generalizes well to both single-vessel and multi-vessel encounters by avoiding rule-based conditions. The performance of this ASV planning strategy is evaluated in simulation against other baseline planners, and the results of on-water testing with a 29-ft ASV demonstrate that the approach is scalable to real systems. Beyond developing improved path planning frameworks, this research also explores methods for improved testing and evaluation of black-box autonomous systems. Statistical learning techniques such as adaptive scenario generation and unsupervised clustering are used to extract the failure modes of the autonomy from large-scale simulation datasets. Subsequently, changes in these failure modes are tracked in a novel form of performance-based regression testing. The effectiveness of this testing framework is demonstrated on the aforementioned ASV planner by discovering several types of unexpected failures

Belief State Planning for Autonomous Driving: Planning with Interaction, Uncertain Prediction and Uncertain Perception

This thesis presents a behavior planning algorithm for automated driving in urban environments with an uncertain and dynamic nature. The uncertainty in the environment arises by the fact that the intentions as well as the future trajectories of the surrounding drivers cannot be measured directly but can only be estimated in a probabilistic fashion. Even the perception of objects is uncertain due to sensor noise or possible occlusions. When driving in such environments, the autonomous car must predict the behavior of the other drivers and plan safe, comfortable and legal trajectories. Planning such trajectories requires robust decision making when several high-level options are available for the autonomous car. Current planning algorithms for automated driving split the problem into different subproblems, ranging from discrete, high-level decision making to prediction and continuous trajectory planning. This separation of one problem into several subproblems, combined with rule-based decision making, leads to sub-optimal behavior. This thesis presents a global, closed-loop formulation for the motion planning problem which intertwines action selection and corresponding prediction of the other agents in one optimization problem. The global formulation allows the planning algorithm to make the decision for certain high-level options implicitly. Furthermore, the closed-loop manner of the algorithm optimizes the solution for various, future scenarios concerning the future behavior of the other agents. Formulating prediction and planning as an intertwined problem allows for modeling interaction, i.e. the future reaction of the other drivers to the behavior of the autonomous car. The problem is modeled as a partially observable Markov decision process (POMDP) with a discrete action and a continuous state and observation space. The solution to the POMDP is a policy over belief states, which contains different reactive plans for possible future scenarios. Surrounding drivers are modeled with interactive, probabilistic agent models to account for their prediction uncertainty. The field of view of the autonomous car is simulated ahead over the whole planning horizon during the optimization of the policy. Simulating the possible, corresponding, future observations allows the algorithm to select actions that actively reduce the uncertainty of the world state. Depending on the scenario, the behavior of the autonomous car is optimized in (combined lateral and) longitudinal direction. The algorithm is formulated in a generic way and solved online, which allows for applying the algorithm on various road layouts and scenarios. While such a generic problem formulation is intractable to solve exactly, this thesis demonstrates how a sufficiently good approximation to the optimal policy can be found online. The problem is solved by combining state of the art Monte Carlo tree search algorithms with near-optimal, domain specific roll-outs. The algorithm is evaluated in scenarios such as the crossing of intersections under unknown intentions of other crossing vehicles, interactive lane changes in narrow gaps and decision making at intersections with large occluded areas. It is shown that the behavior of the closed-loop planner is less conservative than comparable open-loop planners. More precisely, it is even demonstrated that the policy enables the autonomous car to drive in a similar way as an omniscient planner with full knowledge of the scene. It is also demonstrated how the autonomous car executes actions to actively gather more information about the surrounding and to reduce the uncertainty of its belief state

Game-Theoretic Safety Assurance for Human-Centered Robotic Systems

In order for autonomous systems like robots, drones, and self-driving cars to be reliably introduced into our society, they must have the ability to actively account for safety during their operation. While safety analysis has traditionally been conducted offline for controlled environments like cages on factory floors, the much higher complexity of open, human-populated spaces like our homes, cities, and roads makes it unviable to rely on common design-time assumptions, since these may be violated once the system is deployed. Instead, the next generation of robotic technologies will need to reason about safety online, constructing high-confidence assurances informed by ongoing observations of the environment and other agents, in spite of models of them being necessarily fallible.This dissertation aims to lay down the necessary foundations to enable autonomous systems to ensure their own safety in complex, changing, and uncertain environments, by explicitly reasoning about the gap between their models and the real world. It first introduces a suite of novel robust optimal control formulations and algorithmic tools that permit tractable safety analysis in time-varying, multi-agent systems, as well as safe real-time robotic navigation in partially unknown environments; these approaches are demonstrated on large-scale unmanned air traffic simulation and physical quadrotor platforms. After this, it draws on Bayesian machine learning methods to translate model-based guarantees into high-confidence assurances, monitoring the reliability of predictive models in light of changing evidence about the physical system and surrounding agents. This principle is first applied to a general safety framework allowing the use of learning-based control (e.g. reinforcement learning) for safety-critical robotic systems such as drones, and then combined with insights from cognitive science and dynamic game theory to enable safe human-centered navigation and interaction; these techniques are showcased on physical quadrotors—flying in unmodeled wind and among human pedestrians—and simulated highway driving. The dissertation ends with a discussion of challenges and opportunities ahead, including the bridging of safety analysis and reinforcement learning and the need to ``close the loop'' around learning and adaptation in order to deploy increasingly advanced autonomous systems with confidence

Motion Planning for Autonomous Vehicles in Partially Observable Environments

Unsicherheiten, welche aus Sensorrauschen oder nicht beobachtbaren Manöverintentionen anderer Verkehrsteilnehmer resultieren, akkumulieren sich in der Datenverarbeitungskette eines autonomen Fahrzeugs und führen zu einer unvollständigen oder fehlinterpretierten Umfeldrepräsentation. Dadurch weisen Bewegungsplaner in vielen Fällen ein konservatives Verhalten auf. Diese Dissertation entwickelt zwei Bewegungsplaner, welche die Defizite der vorgelagerten Verarbeitungsmodule durch Ausnutzung der Reaktionsfähigkeit des Fahrzeugs kompensieren. Diese Arbeit präsentiert zuerst eine ausgiebige Analyse über die Ursachen und Klassifikation der Unsicherheiten und zeigt die Eigenschaften eines idealen Bewegungsplaners auf. Anschließend befasst sie sich mit der mathematischen Modellierung der Fahrziele sowie den Randbedingungen, welche die Sicherheit gewährleisten. Das resultierende Planungsproblem wird mit zwei unterschiedlichen Methoden in Echtzeit gelöst: Zuerst mit nichtlinearer Optimierung und danach, indem es als teilweise beobachtbarer Markov-Entscheidungsprozess (POMDP) formuliert und die Lösung mit Stichproben angenähert wird. Der auf nichtlinearer Optimierung basierende Planer betrachtet mehrere Manöveroptionen mit individuellen Auftrittswahrscheinlichkeiten und berechnet daraus ein Bewegungsprofil. Er garantiert Sicherheit, indem er die Realisierbarkeit einer zufallsbeschränkten Rückfalloption gewährleistet. Der Beitrag zum POMDP-Framework konzentriert sich auf die Verbesserung der Stichprobeneffizienz in der Monte-Carlo-Planung. Erstens werden Informationsbelohnungen definiert, welche die Stichproben zu Aktionen führen, die eine höhere Belohnung ergeben. Dabei wird die Auswahl der Stichproben für das reward-shaped Problem durch die Verwendung einer allgemeinen Heuristik verbessert. Zweitens wird die Kontinuität in der Reward-Struktur für die Aktionsauswahl ausgenutzt und dadurch signifikante Leistungsverbesserungen erzielt. Evaluierungen zeigen, dass mit diesen Planern große Erfolge in Fahrversuchen und Simulationsstudien mit komplexen Interaktionsmodellen erreicht werden

Qualitative Spatial and Temporal Reasoning based on And/Or Linear Programming An approach to partially grounded qualitative spatial reasoning

Acting intelligently in dynamic environments involves anticipating surrounding processes, for example to foresee a dangerous situation or acceptable social behavior. Knowledge about spatial configurations and how they develop over time enables intelligent robots to safely navigate by reasoning about possible actions. The seamless connection of high-level deliberative processes to perception and action selection remains a challenge though. Moreover, an integration should allow the robot to build awareness of these processes as in reality there will be misunderstandings a robot should be able to respond to. My aim is to verify that actions selected by the robot do not violate navigation or safety regulations and thereby endanger the robot or others. Navigation rules specified qualitatively allow an autonomous agent to consistently combine all rules applicable in a context. Within this thesis, I develop a formal, symbolic representation of right-of-way-rules based on a qualitative spatial representation. This cumulative dissertation consists of 5 peer-reviewed papers and 1 manuscript under review. The contribution of this thesis is an approach to represent navigation patterns based on qualitative spatio-temporal representation and the development of corresponding effective sound reasoning techniques. The approach is based on a spatial logic in the sense of Aiello, Pratt-Hartmann, and van Benthem. This logic has clear spatial and temporal semantics and I demonstrate how it allows various navigation rules and social conventions to be represented. I demonstrate the applicability of the developed method in three different areas, an autonomous robotic system in an industrial setting, an autonomous sailing boat, and a robot that should act politely by adhering to social conventions. In all three settings, the navigation behavior is specified by logic formulas. Temporal reasoning is performed via model checking. An important aspect is that a logic symbol, such as \emph{turn left}, comprises a family of movement behaviors rather than a single pre-specified movement command. This enables to incorporate the current spatial context, the possible changing kinematics of the robotic system, and so on without changing a single formula. Additionally, I show that the developed approach can be integrated into various robotic software architectures. Further, an answer to three long standing questions in the field of qualitative spatial reasoning is presented. Using generalized linear programming as a unifying basis for reasoning, one can jointly reason about relations from different qualitative calculi. Also, concrete entities (fixed points, regions fixed in shape and/or position, etc.) can be mixed with free variables. In addition, a realization of qualitative spatial description can be calculated, i.e., a specific instance/example. All three features are important for applications but cannot be handled by other techniques. I advocate the use of And/Or trees to facilitate efficient reasoning and I show the feasibility of my approach. Last but not least, I investigate a fourth question, how to integrate And/Or trees with linear temporal logic, to enable spatio-temporal reasoning

Dual Loop Rider Control of a Dynamic Motorcycle Riding Simulator

Compared to the automotive industry, the use of simulators in the motorcycle domain is negligible as for their lack of usability and accessibility. According to the state-of-the-art, it is e.g. not possible for motorcyclists to intuitively control a high-fidelity dynamic motorcycle riding simulator when getting in contact with it for the first time. There are four main reasons for the insufficient simulation quality of dynamic motorcycle riding simulators: ▪ The instability of single-track vehicles at low speed, ▪ The steering force-feedback with highly velocity-dependent behavior, ▪ Motion-simulation (high dynamics, roll angle, direct contact to the environment), ▪ The specific influence of the rider to vehicle dynamics (incl. rider motion). The last bullet point is peculiar for motorcycles and dynamic motorcycle riding simulators in comparison with other vehicle simulators, as motorcycles are significantly affected in their dynamics by the rider’s body motion. However, up until today, almost no special emphasis has been put on the consideration of rider motion on dynamic motorcycle riding simulators. In this thesis, a motorcycle riding simulator is designed, constructed and put into operation. The focus here is attaching a real rider to a virtual motorcycle. Based on a commercially available multi-body-simulation model, a simulator architecture is designed, that allows to control the virtual motorcycle not only by steering, but by rider leaning as well. This is realized by determining the so-called rider induced roll torque, that allows a holistic measurement of the apparent coupling forces between rider and simulator mockup. Performance measures and study concepts are developed that allow to rate the system. In expert and participant studies, the influence of the system on the riding behavior of the simulator is investigated. It is shown that the rider motion determination allows realistic control inputs and has a positive effect on the stabilization at various velocities. The feedback of the rider induced roll torque to the virtual dynamics model allows study participants to control the virtual motorcycle more intuitively. The vehicle states during cornering are affected as expected from real riding. First results indicate that it becomes easier for naïve study participants to access the simulator in first-contact scenarios. The achieved improvements regarding the rideability of the simulator however do not suffice to overcome the abovementioned challenges to a degree that allows for a completely intuitive interaction with the simulator throughout the whole dynamic range

Ein Beitrag zur taktischen Verhaltensplanung für Fahrstreifenwechsel bei automatisierten Fahrzeugen

Automated driving within one lane is a fascinating experience. Yet, it is even more interesting to go a step ahead: Making automated lane changes without human driver interaction. This thesis presents a concept and implementation demonstrated in "Jack", the Audi A7 piloted driving concept vehicle. Given that automated driving is in the media every other day already, why is it still such a big issue to do tactical behavior planning for automated vehicles? It is one of the core areas where it is surprisingly obvious why humans are currently so much smarter than machines: Tactical driving behavior planning is a social task that requires cooperation, intention recognition, and complex situation assessment. Without complex cognitive capabilities in today's automated vehicles, it is core of this thesis to find simple algorithms that pretend intelligence in behavior planning. In fact, such behavior planning in automated driving is a constant trade-off between utility and risk: The vehicle has to balance value dimensions such as safety, legality, mobility, and additional aspects like creating user and third party satisfaction. This thesis provides a framework to boil down such abstract dimensions into a working implementation. Several of the foundations for this thesis were developed as part of the Stadtpilot project at TU Braunschweig. While there has been plenty of research on concepts being tested in perfect, simulated worlds only, the approaches in this thesis have been implemented and evaluated in real world traffic with uncertain and imperfect sensor data. The implementation has been tested, tweaked, and used in "Jack" for more than 50,000 km of automated driving in everyday traffic.Automatisiertes Fahren innerhalb eines Fahrstreifens ist eine faszinierende Erfahrung. Noch spannender ist es jedoch noch einen Schritt weiter zu gehen: Auch Fahrstreifenwechsel automatisiert auszuführen, ohne Interaktion mit einem Menschen als Fahrer. In dieser Dissertation wird hierfür ein Konzept und dessen Umsetzung in „Jack“ präsentiert, dem Audi A7 piloted driving concept Fahrzeug. Automatisiertes Fahren ist aktuell in den Medien in aller Munde. Warum ist es dennoch eine große Herausforderung taktische Verhaltensplanung für automatisierte Fahrzeuge wirklich umzusetzen? Es ist einer der Kernbereiche, in denen offensichtlich wird, warum Menschen aktuell Maschinen im Straßenverkehr noch weitaus überlegen sind: Taktische Verhaltensplanung ist eine soziale Aufgabe, welche Kooperation, das Erkennen von Absichten und der Bewertung komplexer Situationen bedarf. Mangels wirklicher kognitiver Fähigkeiten in den heutigen automatisierten Fahrzeugen ist es Kern dieser Dissertation Algorithmen zu finden, welche zumindest den Eindruck intelligenter Verhaltensplanung erzeugen. Eine solche Verhaltensplanung ist ein permanentes Abwägen von Nutzen und Risiken. Das Fahrzeug muss permanent Entscheidungen im Spannungsfeld zwischen Sicherheit, Legalität, Mobilität und weiten Aspekten wie Nutzerzufriedenheit und Zufriedenheit Dritter treffen. In dieser Dissertation wird ein Konzept entwickelt, um solche abstrakten Entscheidungsdimensionen in ein implementierbares Konzept herunterzubrechen. Viele Grundlagen dafür wurden im Rahmen des Stadtpilot Projekts der TU Braunschweig erarbeitet. In vorausgehenden Arbeiten wurden bereits viele Ansätze entwickelt und auf Basis von perfekten, simulierten Daten evaluiert. Der in dieser Arbeit präsentierte Ansatz ist in der Lage mit unsicherheits- und fehlerbehafteten Messdaten umzugehen. Der Ansatz aus dieser Dissertation wurde in dem automatisiert fahrenden Fahrzeug „Jack“ implementiert und bereits über 50.000 km im normalen Straßenverkehr genutzt und getestet