Foresighted People Finding and Following

Mobile service robots are needed in several applications (e.g., transportation systems, autonomous shopping carts, household activities ... etc). In such scenarios the robot aids the user with tasks that require the robot to move freely across the environment in addition to direct interaction at certain times. Therefore, such a robot needs a strategy to quickly find the user whenever needed, in addition to a strategy that enables the robot to reason about the user's intended destination to be able to follow him in a foresighted manner if the user needs its help at that destination. In this dissertation, we tackle each of those problems separately in a divide and conquer manner. We present an approach to learn optimal navigation actions for assistance tasks in which the robot aims at efficiently reaching the final navigation goal of a human where service has to be provided. Always following the human at a close distance might hereby result in inefficient trajectories, since people regularly do not move on the shortest path to their destination (e.g., they may move to grab the phone or make a note). Therefore, a service robot should infer the human's intended navigation goal and compute its own motion based on that prediction. We propose to perform a prediction about the human's future movements and use this information in a reinforcement learning framework to generate foresighted navigation actions for the robot. Since frequent occlusions of the human will occur due to obstacles and the robot's constrained field of view, the estimate about the humans's position and the prediction of the next destination are affected by uncertainty. Our approach deals with such situations by explicitly considering occlusions in the reward function such that the robot automatically considers to execute actions to get the human in its field of view. We show in simulated and real-world experiments that our technique leads to significantly shorter paths compared to an approach in which the robot always tries to closely follow the user and, additionally, can handle occlusions. On the other side, an autonomous robot that directly helps users with certain tasks often first has to quickly find a user, especially when this person moves around frequently. A search method that relies on a greedy approach that do not perform any predictions about the user's most likely location, even when it is provided with background information about the frequently visited destinations of the user, might not be the best option. In this dissertation, we propose to compute the likelihood of the user's observability at each possible location in the environment based on simulations that rely on hidden Markov model based predictions. As the robot needs time to reach the search locations, we take this time into account as well as the visibility constraints. In this way we aim at selecting effective search locations for the robot to find the user as fast as possible. As our experiments in various simulated environments show, our approach leads to significantly shorter search times compared to the greedy approach

Spatiotemporal Attention Enhances Lidar-Based Robot Navigation in Dynamic Environments

Foresighted robot navigation in dynamic indoor environments with cost-efficient hardware necessitates the use of a lightweight yet dependable controller. So inferring the scene dynamics from sensor readings without explicit object tracking is a pivotal aspect of foresighted navigation among pedestrians. In this paper, we introduce a spatiotemporal attention pipeline for enhanced navigation based on 2D lidar sensor readings. This pipeline is complemented by a novel lidar-state representation that emphasizes dynamic obstacles over static ones. Subsequently, the attention mechanism enables selective scene perception across both space and time, resulting in improved overall navigation performance within dynamic scenarios. We thoroughly evaluated the approach in different scenarios and simulators, finding good generalization to unseen environments. The results demonstrate outstanding performance compared to state-of-the-art methods, thereby enabling the seamless deployment of the learned controller on a real robot

Autonomous Navigation with Deep Reinforcement Learning in Carla Simulator

With the rapid development of autonomous driving and artificial intelligence technology, end-to-end autonomous driving technology has become a research hotspot. This thesis aims to explore the application of deep reinforcement learning in the realizing of end-to-end autonomous driving. We built a deep reinforcement learning virtual environment in the Carla simulator, and based on it, we trained a policy model to control a vehicle along a preplanned route. For the selection of the deep reinforcement learning algorithms, we have used the Proximal Policy Optimization algorithm due to its stable performance. Considering the complexity of end-to-end autonomous driving, we have also carefully designed a comprehensive reward function to train the policy model more efficiently. The model inputs for this study are of two types: firstly, real-time road information and vehicle state data obtained from the Carla simulator, and secondly, real-time images captured by the vehicle's front camera. In order to understand the influence of different input information on the training effect and model performance, we conducted a detailed comparative analysis. The test results showed that the accuracy and significance of the information has a significant impact on the learning effect of the agent, which in turn has a direct impact on the performance of the model. Through this study, we have not only confirmed the potential of deep reinforcement learning in the field of end-to-end autonomous driving, but also provided an important reference for future research and development of related technologies

Situation-based Risk Evaluation and Behavior Planning

The presented dissertation addresses the problem of risk evaluation and behavior planning for future intelligent Advanced Driver Assistance Systems (ADAS). For this purpose, a novel framework for situation-based risk evaluation and behavior planning, targeting highly automated driving, is presented. After properly sensing the current scene, including the current road topology and other traffic participants, the proposed framework first estimates and predicts the future behavior of all involved entities comprising a situation classification and trajectory prediction step. This is then followed by the generation of the own future behavior in a behavior planning step which is based on an evaluation of possible ego behavior alternatives in terms of risk and utility considerations. The future behavior is planned in a way to find a tradeoff between the expected future risk and utility. Inner-city traffic scenarios in particular are usually complex and of high uncertainty, considering measurements as well as behavioral decisions. To reduce the complexity, similar behavior alternatives are clustered and represented by prototypical behavior patterns using so-called situations. A novel situation classification approach is proposed to estimate how good a situation matches with the actual behaviors. This approach is based on a comparison of the prototypically predicted trajectories of the considered situations with the actual measured trajectories. For this purpose a novel measure for spatio-temporal trajectory similarity, based on the evaluation of longitudinal and lateral spatio-temporal distance, is derived. The situation classification system is used to detect incorrect and critical traffic behaviors, especially in scenarios with a disregard of right-of-way. Evaluating the system using real-world crash cases reveals that it is able to warn the driver reliably of an upcoming crash, with sufficient time to initiate a suitable evasive behavior. For the prediction of situation-dependent prototypical scene evolution patterns, the interaction-aware Foresighted Driver Model (FDM) is applied in a forward simulation of a sensed scene under different situation-dependent behavioral assumptions. The proposed FDM is a novel, time continuous driver model for the simulation and prediction of freeway and urban traffic. Based on the general risk evaluation and behavior planning framework developed in this thesis, the driver model equations are introduced from the assumption that a driver tries to balance predictive risk (e.g. due to possible collisions along its route) with utility (e.g. the time required to travel, smoothness of ride, etc.). For this purpose, a computationally inexpensive, approximate risk model targeting only risk maxima and a gradient descent-based behavior generation is applied. It is shown, how such a model can be used to simulate and predict driving behavior with a similar performance compared to full behavior planning models. The FDM is applicable to a wide range of different scenarios, e.g. intersection or highway-accessing scenarios, with the consideration of an arbitrary number of traffic entities. Thus, the FDM generalizes and reaches beyond state-of-the-art driver models. Complex traffic situations require the estimation of future behavior alternatives in terms of predictive risks. Risk assessment has to be driven from the knowledge that the acting scene entity requires to evaluate the own future behavior. Based on the predicted future dynamics of traffic scene entities, an approach is presented, where a continuous, probabilistic model for future risk is used to build so-called predictive risk maps. These maps indicate how risky a certain ego behavior will be at different future times, so that they can be used to directly plan the best possible future behavior. The behavior in complex scenarios differs strongly, depending on the actually occurring situation. However, sensory measurements of the ego- and other involved entities' states as well as the prediction of possible future states are generally of high uncertainty. As a consequence, the current driving situation can only be approximated. Additionally, a situation can change very quickly, e.g. if a traffic participant suddenly changes its behavior. In this thesis an approach is proposed, how to plan a safe, but still efficient future behavior under consideration of multiple possible situations with different occurrence probabilities. In several traffic scenarios comprising simulated as well as recorded real-world data, it is shown that the approach generates an efficient behavior for situations which are likely to occur, while generating a plan B to safely deal with improbable but risky situations

Belief State Planning for Autonomous Driving: Planning with Interaction, Uncertain Prediction and Uncertain Perception

This thesis presents a behavior planning algorithm for automated driving in urban environments with an uncertain and dynamic nature. The uncertainty in the environment arises by the fact that the intentions as well as the future trajectories of the surrounding drivers cannot be measured directly but can only be estimated in a probabilistic fashion. Even the perception of objects is uncertain due to sensor noise or possible occlusions. When driving in such environments, the autonomous car must predict the behavior of the other drivers and plan safe, comfortable and legal trajectories. Planning such trajectories requires robust decision making when several high-level options are available for the autonomous car. Current planning algorithms for automated driving split the problem into different subproblems, ranging from discrete, high-level decision making to prediction and continuous trajectory planning. This separation of one problem into several subproblems, combined with rule-based decision making, leads to sub-optimal behavior. This thesis presents a global, closed-loop formulation for the motion planning problem which intertwines action selection and corresponding prediction of the other agents in one optimization problem. The global formulation allows the planning algorithm to make the decision for certain high-level options implicitly. Furthermore, the closed-loop manner of the algorithm optimizes the solution for various, future scenarios concerning the future behavior of the other agents. Formulating prediction and planning as an intertwined problem allows for modeling interaction, i.e. the future reaction of the other drivers to the behavior of the autonomous car. The problem is modeled as a partially observable Markov decision process (POMDP) with a discrete action and a continuous state and observation space. The solution to the POMDP is a policy over belief states, which contains different reactive plans for possible future scenarios. Surrounding drivers are modeled with interactive, probabilistic agent models to account for their prediction uncertainty. The field of view of the autonomous car is simulated ahead over the whole planning horizon during the optimization of the policy. Simulating the possible, corresponding, future observations allows the algorithm to select actions that actively reduce the uncertainty of the world state. Depending on the scenario, the behavior of the autonomous car is optimized in (combined lateral and) longitudinal direction. The algorithm is formulated in a generic way and solved online, which allows for applying the algorithm on various road layouts and scenarios. While such a generic problem formulation is intractable to solve exactly, this thesis demonstrates how a sufficiently good approximation to the optimal policy can be found online. The problem is solved by combining state of the art Monte Carlo tree search algorithms with near-optimal, domain specific roll-outs. The algorithm is evaluated in scenarios such as the crossing of intersections under unknown intentions of other crossing vehicles, interactive lane changes in narrow gaps and decision making at intersections with large occluded areas. It is shown that the behavior of the closed-loop planner is less conservative than comparable open-loop planners. More precisely, it is even demonstrated that the policy enables the autonomous car to drive in a similar way as an omniscient planner with full knowledge of the scene. It is also demonstrated how the autonomous car executes actions to actively gather more information about the surrounding and to reduce the uncertainty of its belief state

Belief State Planning for Autonomous Driving: Planning with Interaction, Uncertain Prediction and Uncertain Perception

This work presents a behavior planning algorithm for automated driving in urban environments with an uncertain and dynamic nature. The algorithm allows to consider the prediction uncertainty (e.g. different intentions), perception uncertainty (e.g. occlusions) as well as the uncertain interactive behavior of the other agents explicitly. Simulating the most likely future scenarios allows to find an optimal policy online that enables non-conservative planning under uncertainty

Navigation behavior design and representations for a people aware mobile robot system

There are millions of robots in operation around the world today, and almost all of them operate on factory floors in isolation from people. However, it is now becoming clear that robots can provide much more value assisting people in daily tasks in human environments. Perhaps the most fundamental capability for a mobile robot is navigating from one location to another. Advances in mapping and motion planning research in the past decades made indoor navigation a commodity for mobile robots. Yet, questions remain on how the robots should move around humans. This thesis advocates the use of semantic maps and spatial rules of engagement to enable non-expert users to effortlessly interact with and control a mobile robot. A core concept explored in this thesis is the Tour Scenario, where the task is to familiarize a mobile robot to a new environment after it is first shipped and unpacked in a home or office setting. During the tour, the robot follows the user and creates a semantic representation of the environment. The user labels objects, landmarks and locations by performing pointing gestures and using the robot's user interface. The spatial semantic information is meaningful to humans, as it allows providing commands to the robot such as ``bring me a cup from the kitchen table". While the robot is navigating towards the goal, it should not treat nearby humans as obstacles and should move in a socially acceptable manner. Three main navigation behaviors are studied in this work. The first behavior is the point-to-point navigation. The navigation planner presented in this thesis borrows ideas from human-human spatial interactions, and takes into account personal spaces as well as reactions of people who are in close proximity to the trajectory of the robot. The second navigation behavior is person following. After the description of a basic following behavior, a user study on person following for telepresence robots is presented. Additionally, situation awareness for person following is demonstrated, where the robot facilitates tasks by predicting the intent of the user and utilizing the semantic map. The third behavior is person guidance. A tour-guide robot is presented with a particular application for visually impaired users.Ph.D

Exploration of smart infrastructure for drivers of autonomous vehicles

The connection between vehicles and infrastructure is an integral part of providing autonomous vehicles information about the environment. Autonomous vehicles need to be safe and users need to trust their driving decision. When smart infrastructure information is integrated into the vehicle, the driver needs to be informed in an understandable manner what the smart infrastructure detected. Nevertheless, interactions that benefit from smart infrastructure have not been the focus of research, leading to knowledge gaps in the integration of smart infrastructure information in the vehicle. For example, it is unclear, how the information from two complex systems can be presented, and if decisions are made, how these can be explained. Enriching the data of vehicles with information from the infrastructure opens unexplored opportunities. Smart infrastructure provides vehicles with information to predict traffic flow and traffic events. Additionally, it has information about traffic events in several kilometers distance and thus enables a look ahead on a traffic situation, which is not in the immediate view of drivers. We argue that this smart infrastructure information can be used to enhance the driving experience. To achieve this, we explore designing novel interactions, providing warnings and visualizations about information that is out of the view of the driver, and offering explanations for the cause of changed driving behavior of the vehicle. This thesis focuses on exploring the possibilities of smart infrastructure information with a focus on the highway. The first part establishes a design space for 3D in-car augmented reality applications that profit from smart infrastructure information. Through the input of two focus groups and a literature review, use cases are investigated that can be introduced in the vehicle's interaction interface which, among others, rely on environment information. From those, a design space that can be used to design novel in-car applications is derived. The second part explores out-of-view visualizations before and during take over requests to increase situation awareness. With three studies, different visualizations for out-of-view information are implemented in 2D, stereoscopic 3D, and augmented reality. Our results show that visualizations improve the situation awareness about critical events in larger distances during take over request situations. In the third part, explanations are designed for situations in which the vehicle drives unexpectedly due to unknown reasons. Since smart infrastructure could provide connected vehicles with out-of-view or cloud information, the driving maneuver of the vehicle might remain unclear to the driver. Therefore, we explore the needs of drivers in those situations and derive design recommendations for an interface which displays the cause for the unexpected driving behavior. This thesis answers questions about the integration of environment information in vehicles'. Three important aspects are explored, which are essential to consider when implementing use cases with smart infrastructure in mind. It enables to design novel interactions, provides insights on how out-of-view visualizations can improve the drivers' situation awareness and explores unexpected driving situations and the design of explanations for them. Overall, we have shown how infrastructure and connected vehicle information can be introduced in vehicles' user interface and how new technology such as augmented reality glasses can be used to improve the driver's perception of the environment.Autonome Fahrzeuge werden immer mehr in den alltäglichen Verkehr integriert. Die Verbindung von Fahrzeugen mit der Infrastruktur ist ein wesentlicher Bestandteil der Bereitstellung von Umgebungsinformationen in autonome Fahrzeugen. Die Erweiterung der Fahrzeugdaten mit Informationen der Infrastruktur eröffnet ungeahnte Möglichkeiten. Intelligente Infrastruktur übermittelt verbundenen Fahrzeugen Informationen über den prädizierten Verkehrsfluss und Verkehrsereignisse. Zusätzlich können Verkehrsgeschehen in mehreren Kilometern Entfernung übermittelt werden, wodurch ein Vorausblick auf einen Bereich ermöglicht wird, der für den Fahrer nicht unmittelbar sichtbar ist. Mit dieser Dissertation wird gezeigt, dass Informationen der intelligenten Infrastruktur benutzt werden können, um das Fahrerlebnis zu verbessern. Dies kann erreicht werden, indem innovative Interaktionen gestaltet werden, Warnungen und Visualisierungen über Geschehnisse außerhalb des Sichtfelds des Fahrers vermittelt werden und indem Erklärungen über den Grund eines veränderten Fahrzeugverhaltens untersucht werden. Interaktionen, welche von intelligenter Infrastruktur profitieren, waren jedoch bisher nicht im Fokus der Forschung. Dies führt zu Wissenslücken bezüglich der Integration von intelligenter Infrastruktur in das Fahrzeug. Diese Dissertation exploriert die Möglichkeiten intelligenter Infrastruktur, mit einem Fokus auf die Autobahn. Der erste Teil erstellt einen Design Space für Anwendungen von augmentierter Realität (AR) in 3D innerhalb des Autos, die unter anderem von Informationen intelligenter Infrastruktur profitieren. Durch das Ergebnis mehrerer Studien werden Anwendungsfälle in einem Katalog gesammelt, welche in die Interaktionsschnittstelle des Autos einfließen können. Diese Anwendungsfälle bauen unter anderem auf Umgebungsinformationen. Aufgrund dieser Anwendungen wird der Design Space entwickelt, mit Hilfe dessen neuartige Anwendungen für den Fahrzeuginnenraum entwickelt werden können. Der zweite Teil exploriert Visualisierungen für Verkehrssituationen, die außerhalb des Sichtfelds des Fahrers sind. Es wird untersucht, ob durch diese Visualisierungen der Fahrer besser auf ein potentielles Übernahmeszenario vorbereitet wird. Durch mehrere Studien wurden verschiedene Visualisierungen in 2D, stereoskopisches 3D und augmentierter Realität implementiert, die Szenen außerhalb des Sichtfelds des Fahrers darstellen. Diese Visualisierungen verbessern das Situationsbewusstsein über kritische Szenarien in einiger Entfernung während eines Übernahmeszenarios. Im dritten Teil werden Erklärungen für Situationen gestaltet, in welchen das Fahrzeug ein unerwartetes Fahrmanöver ausführt. Der Grund des Fahrmanövers ist dem Fahrer dabei unbekannt. Mit intelligenter Infrastruktur verbundene Fahrzeuge erhalten Informationen, die außerhalb des Sichtfelds des Fahrers liegen oder von der Cloud bereit gestellt werden. Dadurch könnte der Grund für das unerwartete Fahrverhalten unklar für den Fahrer sein. Daher werden die Bedürfnisse des Fahrers in diesen Situationen erforscht und Empfehlungen für die Gestaltung einer Schnittstelle, die Erklärungen für das unerwartete Fahrverhalten zur Verfügung stellt, abgeleitet. Zusammenfassend wird gezeigt wie Daten der Infrastruktur und Informationen von verbundenen Fahrzeugen in die Nutzerschnittstelle des Fahrzeugs implementiert werden können. Zudem wird aufgezeigt, wie innovative Technologien wie AR Brillen, die Wahrnehmung der Umgebung des Fahrers verbessern können. Durch diese Dissertation werden Fragen über Anwendungsfälle für die Integration von Umgebungsinformationen in Fahrzeugen beantwortet. Drei wichtige Themengebiete wurden untersucht, welche bei der Betrachtung von Anwendungsfällen der intelligenten Infrastruktur essentiell sind. Durch diese Arbeit wird die Gestaltung innovativer Interaktionen ermöglicht, Einblicke in Visualisierungen von Informationen außerhalb des Sichtfelds des Fahrers gegeben und es wird untersucht, wie Erklärungen für unerwartete Fahrsituationen gestaltet werden können

Ein Beitrag zur taktischen Verhaltensplanung für Fahrstreifenwechsel bei automatisierten Fahrzeugen

Automated driving within one lane is a fascinating experience. Yet, it is even more interesting to go a step ahead: Making automated lane changes without human driver interaction. This thesis presents a concept and implementation demonstrated in "Jack", the Audi A7 piloted driving concept vehicle. Given that automated driving is in the media every other day already, why is it still such a big issue to do tactical behavior planning for automated vehicles? It is one of the core areas where it is surprisingly obvious why humans are currently so much smarter than machines: Tactical driving behavior planning is a social task that requires cooperation, intention recognition, and complex situation assessment. Without complex cognitive capabilities in today's automated vehicles, it is core of this thesis to find simple algorithms that pretend intelligence in behavior planning. In fact, such behavior planning in automated driving is a constant trade-off between utility and risk: The vehicle has to balance value dimensions such as safety, legality, mobility, and additional aspects like creating user and third party satisfaction. This thesis provides a framework to boil down such abstract dimensions into a working implementation. Several of the foundations for this thesis were developed as part of the Stadtpilot project at TU Braunschweig. While there has been plenty of research on concepts being tested in perfect, simulated worlds only, the approaches in this thesis have been implemented and evaluated in real world traffic with uncertain and imperfect sensor data. The implementation has been tested, tweaked, and used in "Jack" for more than 50,000 km of automated driving in everyday traffic.Automatisiertes Fahren innerhalb eines Fahrstreifens ist eine faszinierende Erfahrung. Noch spannender ist es jedoch noch einen Schritt weiter zu gehen: Auch Fahrstreifenwechsel automatisiert auszuführen, ohne Interaktion mit einem Menschen als Fahrer. In dieser Dissertation wird hierfür ein Konzept und dessen Umsetzung in „Jack“ präsentiert, dem Audi A7 piloted driving concept Fahrzeug. Automatisiertes Fahren ist aktuell in den Medien in aller Munde. Warum ist es dennoch eine große Herausforderung taktische Verhaltensplanung für automatisierte Fahrzeuge wirklich umzusetzen? Es ist einer der Kernbereiche, in denen offensichtlich wird, warum Menschen aktuell Maschinen im Straßenverkehr noch weitaus überlegen sind: Taktische Verhaltensplanung ist eine soziale Aufgabe, welche Kooperation, das Erkennen von Absichten und der Bewertung komplexer Situationen bedarf. Mangels wirklicher kognitiver Fähigkeiten in den heutigen automatisierten Fahrzeugen ist es Kern dieser Dissertation Algorithmen zu finden, welche zumindest den Eindruck intelligenter Verhaltensplanung erzeugen. Eine solche Verhaltensplanung ist ein permanentes Abwägen von Nutzen und Risiken. Das Fahrzeug muss permanent Entscheidungen im Spannungsfeld zwischen Sicherheit, Legalität, Mobilität und weiten Aspekten wie Nutzerzufriedenheit und Zufriedenheit Dritter treffen. In dieser Dissertation wird ein Konzept entwickelt, um solche abstrakten Entscheidungsdimensionen in ein implementierbares Konzept herunterzubrechen. Viele Grundlagen dafür wurden im Rahmen des Stadtpilot Projekts der TU Braunschweig erarbeitet. In vorausgehenden Arbeiten wurden bereits viele Ansätze entwickelt und auf Basis von perfekten, simulierten Daten evaluiert. Der in dieser Arbeit präsentierte Ansatz ist in der Lage mit unsicherheits- und fehlerbehafteten Messdaten umzugehen. Der Ansatz aus dieser Dissertation wurde in dem automatisiert fahrenden Fahrzeug „Jack“ implementiert und bereits über 50.000 km im normalen Straßenverkehr genutzt und getestet