Motion Planning for Autonomous Vehicles in Partially Observable Environments

Unsicherheiten, welche aus Sensorrauschen oder nicht beobachtbaren Manöverintentionen anderer Verkehrsteilnehmer resultieren, akkumulieren sich in der Datenverarbeitungskette eines autonomen Fahrzeugs und führen zu einer unvollständigen oder fehlinterpretierten Umfeldrepräsentation. Dadurch weisen Bewegungsplaner in vielen Fällen ein konservatives Verhalten auf. Diese Dissertation entwickelt zwei Bewegungsplaner, welche die Defizite der vorgelagerten Verarbeitungsmodule durch Ausnutzung der Reaktionsfähigkeit des Fahrzeugs kompensieren. Diese Arbeit präsentiert zuerst eine ausgiebige Analyse über die Ursachen und Klassifikation der Unsicherheiten und zeigt die Eigenschaften eines idealen Bewegungsplaners auf. Anschließend befasst sie sich mit der mathematischen Modellierung der Fahrziele sowie den Randbedingungen, welche die Sicherheit gewährleisten. Das resultierende Planungsproblem wird mit zwei unterschiedlichen Methoden in Echtzeit gelöst: Zuerst mit nichtlinearer Optimierung und danach, indem es als teilweise beobachtbarer Markov-Entscheidungsprozess (POMDP) formuliert und die Lösung mit Stichproben angenähert wird. Der auf nichtlinearer Optimierung basierende Planer betrachtet mehrere Manöveroptionen mit individuellen Auftrittswahrscheinlichkeiten und berechnet daraus ein Bewegungsprofil. Er garantiert Sicherheit, indem er die Realisierbarkeit einer zufallsbeschränkten Rückfalloption gewährleistet. Der Beitrag zum POMDP-Framework konzentriert sich auf die Verbesserung der Stichprobeneffizienz in der Monte-Carlo-Planung. Erstens werden Informationsbelohnungen definiert, welche die Stichproben zu Aktionen führen, die eine höhere Belohnung ergeben. Dabei wird die Auswahl der Stichproben für das reward-shaped Problem durch die Verwendung einer allgemeinen Heuristik verbessert. Zweitens wird die Kontinuität in der Reward-Struktur für die Aktionsauswahl ausgenutzt und dadurch signifikante Leistungsverbesserungen erzielt. Evaluierungen zeigen, dass mit diesen Planern große Erfolge in Fahrversuchen und Simulationsstudien mit komplexen Interaktionsmodellen erreicht werden

Resolving uncertainty on the fly: Modeling adaptive driving behavior as active inference

Understanding adaptive human driving behavior, in particular how drivers manage uncertainty, is of key importance for developing simulated human driver models that can be used in the evaluation and development of autonomous vehicles. However, existing traffic psychology models of adaptive driving behavior either lack computational rigor or only address specific scenarios and/or behavioral phenomena. While models developed in the fields of machine learning and robotics can effectively learn adaptive driving behavior from data, due to their black box nature, they offer little or no explanation of the mechanisms underlying the adaptive behavior. Thus, a generalizable, interpretable, computational model of adaptive human driving behavior is still lacking. This paper proposes such a model based on active inference, a behavioral modeling framework originating in computational neuroscience. The model offers a principled solution to how humans trade progress against caution through policy selection based on the single mandate to minimize expected free energy. This casts goal-seeking and information-seeking (uncertainty-resolving) behavior under a single objective function, allowing the model to seamlessly resolve uncertainty as a means to obtain its goals. We apply the model in two apparently disparate driving scenarios that require managing uncertainty, (1) driving past an occluding object and (2) visual time sharing between driving and a secondary task, and show how human-like adaptive driving behavior emerges from the single principle of expected free energy minimization.Comment: 33 pages, 13 figure

PLANNING UNDER UNCERTAINTIES FOR AUTONOMOUS DRIVING ON URBAN ROAD

Ph.DDOCTOR OF PHILOSOPH

Sensor Path Planning for Emitter Localization

The localization of a radio frequency (RF) emitter is relevant in many military and civilian applications. The recent decade has seen a rapid progress in the development of small and mobile unmanned aerial vehicles (UAVs), which offer a way to perform emitter localization autonomously. The path a UAV travels influences the localization significantly, making path planning an important part of a mobile emitter localization system. The topic of this thesis is path planning for a UAV that uses bearing measurements to localize a stationary emitter. Using a directional antenna, the direction towards the target can be determined by the UAV rotating around its own vertical axis. During this rotation the UAV is required to remain at the same position, which induces a trade-off between movement and measurement that influences the optimal trajectories. This thesis derives a novel path planning algorithm for localizing an emitter with a UAV. It improves the current state of the art by providing a localization with defined accuracy in a shorter amount of time compared to other algorithms in simulations. The algorithm uses the policy rollout principle to perform a nonmyopic planning and to incorporate the uncertainty of the estimation process into its decision. The concept of an action selection algorithm for policy rollout is introduced, which allows the use of existing optimization algorithms to effectively search the action space. Multiple action selection algorithms are compared to optimize the speed of the path planning algorithm. Similarly, to reduce computational demand, an adaptive grid-based localizer has been developed. To evaluate the algorithm an experimental system has been built and the algorithm was tested on this system. Based on initial experiments, the path planning algorithm has been modified, including a minimal distance to the emitter and an outlier detection step. The resulting algorithm shows promising results in experimental flights

Monte-Carlo Based Online planning Under Partial Observability : Solving Single and Multi-Agent Problems

This thesis thoroughly explores the integration of statistical and reinforcement learning techniques, aiming to provide fresh perspectives and solutions for enhancing the current state-of-the-art methods considering the capabilities of autonomous agents to perform learning, planning and estimation in an online manner in a single and multi-agent systems context. We aim to address a critical demand in the field, steering away from the prevailing dependence on the application of intensive computational resources and large amounts of data as a requirement to achieve peak performance in our context. Our primary focus centres on studying and refining solutions in the ``online planning under uncertainty'' research area. We have ventured beyond the boundaries of existing literature, pushing our proposals to more complex and challenging problems. As concrete contributions, we introduce three new algorithms: IB-POMCP, an online planning algorithm which uses information entropy to augment a single agent's decision making capabilities; OEATE, a type and parameter estimation method to handle coordination with multiple unknown teammates in cooperative environments; and BAE, a method capable of detecting adverserial agents disguised as teammates in cooperative environments on-the-fly. Our proposals contribute to the evolution of autonomous systems and are supported by empirical and theoretical results. We demonstrate that our new perspectives for agents' reasoning processes can present generic and extendable solutions to diverse scenarios and problems. Finally, during the PhD journey, we have developed and presented to the research community a new framework designed to aggregate relevant baselines and benchmarks for multi-agent systems: the AdLeap-MAS. AdLeap-MAS framework stands out as a novel tool centred on the implementation and simulation of ad-hoc reasoning domains for multi-agent, collaborative, and adversarial contexts. The framework aims to facilitate the execution of experiments and the re-use existing codes across different environments. We provide a user-friendly environment that not only extends the frontiers of our research but also serves as a valuable resource for the research community

Uncertainty and social considerations for mobile assistive robot navigation

An increased interest in mobile robots has been seen over the past years. The wide range of possible applications, from vacuum cleaners to assistant robots, makes such robots an interesting solution to many everyday problems. A key requirement for the mass deployment of such robots is to ensure they can safely navigate around our daily living environments. A robot colliding with or bumping into a person may be, in some contexts, unacceptable. For example, if a robot working around elderly people collides with one of them, it may cause serious injuries. This thesis explores four major components required for effective robot navigation: sensing the static environment, detection and tracking of moving people, obstacle and people avoidance with uncertainty measurement, and basic social navigation considerations. First, to guarantee adherence to basic safety constraints, sensors and algorithms required to measure the complex structure of our daily living environments are explored. Not only do the static components of the environment have to be measured, but so do any people present. A people detection and tracking algorithm, aimed for a crowded environment is proposed, thus enhancing the robot's perception capabilities. Our daily living environments present many inherent sources of uncertainty for robots, one of them arising due to the robot's inability to know people's intentions as they move. To solve this problem, a motion model that assumes unknown long-term intentions is proposed. This is used in conjunction with a novel uncertainty aware local planner to create feasible trajectories. In social situations, the presence of groups of people cannot be neglected when navigating. To avoid the robot interrupting groups of people, it first needs to be able to detect such groups. A group detector is proposed which relies on a set of gaze- and geometric-based features. Avoiding group disruption is finally incorporated into the navigation algorithm by means of taking into account the probability of disrupting a group's activities. The effectiveness of the four different components is evaluated using real world and simulated data, demonstrating the benefits for mobile robot navigation.Open Acces

Planning, Estimation and Control for Mobile Robot Localization with Application to Long-Term Autonomy

There may arise two kinds of challenges in the problem of mobile robot localization; (i) a robot may have an a priori map of its environment, in which case the localization problem boils down to estimating the robot pose relative to a global frame or (ii) no a priori map information is given, in which case a robot may have to estimate a model of its environment and localize within it. In the case of a known map, simultaneous planning while localizing is a crucial ability for operating under uncertainty. We first address this problem by designing a method to dynamically replan while the localization uncertainty or environment map is updated. Extensive simulations are conducted to compare the proposed method with the performance of FIRM (Feedback-based Information RoadMap). However, a shortcoming of this method is its reliance on a Gaussian assumption for the Probability Density Function (pdf) on the robot state. This assumption may be violated during autonomous operation when a robot visits parts of the environment which appear similar to others. Such situations lead to ambiguity in data association between what is seen and the robot’s map leading to a non-Gaussian pdf on the robot state. We address this challenge by developing a motion planning method to resolve situations where ambiguous data associations result in a multimodal hypothesis on the robot state. A Receding Horizon approach is developed, to plan actions that sequentially disambiguate a multimodal belief to achieve tight localization on the correct pose in finite time. In our method, disambiguation is achieved through active data associations by picking target states in the map which allow distinctive information to be observed for each belief mode and creating local feedback controllers to visit the targets. Experiments are conducted for a kidnapped physical ground robot operating in an artificial maze-like environment. The hardest challenge arises when no a priori information is present. In longterm tasks where a robot must drive for long durations before closing loops, our goal is to minimize the localization error growth rate such that; (i) accurate data associations can be made for loop closure, or (ii) in cases where loop closure is not possible, the localization error stays limited within some desired bounds. We analyze this problem and show that accurate heading estimation is key to limiting localization error drift. We make three contributions in this domain. First we present a method for accurate long-term localization using absolute orientation measurements and analyze the underlying structure of the SLAM problem and how it is affected by unbiased heading measurements. We show that consistent estimates over a 100km trajectory are possible and that the error growth rate can be controlled with active data acquisition. Then we study the more general problem when orientation measurements may not be present and develop a SLAM technique to separate orientation and position estimation. We show that our method’s accuracy degrades gracefully compared to the standard non-linear optimization based SLAM approach and avoids catastrophic failures which may occur due a bad initial guess in non-linear optimization. Finally we take our understanding of orientation sensing into the physical world and demonstrate a 2D SLAM technique that leverages absolute orientation sensing based on naturally occurring structural cues. We demonstrate our method using both high-fidelity simulations and a real-world experiment in a 66, 000 square foot warehouse. Empirical studies show that maps generated by our approach never suffer catastrophic failure, whereas existing scan matching based SLAM methods fail ≈ 50% of the time

Simultaneous Search and Monitoring by Unmanned Aerial Vehicles

Although robot search and monitoring are two problems which are normally addressed separately, this work conceives the idea that search and monitoring are both required in realistic applications. A problem of simultaneous search and monitoring (SSM) is studied, which innovatively combines two problems in a synergistic perspective. The single pursuer SSM of randomly moving or evasive targets are studied first, and are extended to the cases with multiple pursuers. The precise mathematical frameworks for this work are POMDP, POSG and Dec-POMDP. They are all intractable and non-scalable. Different approaches are taken in each scenario, to reduce computation cost and achieve online and distributed planning, without significantly undermining the performance. For the single pursuer SSM of randomly moving targets, a novel policy reconstruction method is combined with a heuristic branching rule, to generate a heuristic reactive policy. For the single pursuer SSM of evasive targets, an assumption is made and justified, which simplifies the search evasion game to a dynamic guaranteed search problem. For the multiple-pursuer SSM of randomly moving targets, the partial open-loop feedback control method is originally applied to achieve the cooperation implicitly. For the multiple-pursuer SSM of evasive targets, the assumption made in the single pursuer case also simplifies the cooperative search evasion game to a cooperative dynamic guaranteed search problem. In moderate scenarios, the proposed methods show better performance than baseline methods, and can have practical computation efficiency. The extreme scenarios when SSM does not work are also studied

Active Perception for Autonomous Systems : In a Deep Space Navigation Scenario

Autonomous systems typically pursue certain goals for an extended amount of time in a self-sustainable fashion. To this end, they are equipped with a set of sensors and actuators to perceive certain aspects of the world and thereupon manipulate it in accordance with some given goals. This kind of interaction can be thought of as a closed loop in which a perceive-reason-act process takes place. The bi-directional interface between an autonomous system and the outer world is then given by a sequence of imperfect observations of the world and corresponding controls which are as well imperfectly actuated. To be able to reason in such a setting, it is customary for an autonomous system to maintain a probabilistic state estimate. The quality of the estimate -- or its uncertainty -- is, in turn, dependent on the information acquired within the perceive-reason-act loop described above. Hence, this thesis strives to investigate the question of how to actively steer such a process in order to maximize the quality of the state estimate. The question will be approached by introducing different probabilistic state estimation schemes jointly working on a manifold-based encapsuled state representation. On top of the resultant state estimate different active perception approaches are introduced, which determine optimal actions with respect to uncertainty minimization. The informational value of the particular actions is given by the expected impact of measurements on the uncertainty. The latter can be obtained by different direct and indirect measures, which will be introduced and discussed. The active perception schemes for autonomous systems will be investigated with a focus on two specific deep space navigation scenarios deduced from a potential mining mission to the main asteroid belt. In the first scenario, active perception strategies are proposed, which foster the correctional value of the sensor information acquired within a heliocentric navigation approach. Here, the expected impact of measurements is directly estimated, thus omitting counterfactual updates of the state based on hypothetical actions. Numerical evaluations of this scenario show that active perception is beneficial, i.e., the quality of the state estimate is increased. In addition, it is shown that the more uncertain a state estimate is, the more the value of active perception increases. In the second scenario, active autonomous deep space navigation in the vicinity of asteroids is investigated. A trajectory and a map are jointly estimated by a Graph SLAM algorithm based on measurements of a 3D Flash-LiDAR. The active perception strategy seeks to trade-off the exploration of the asteroid against the localization performance. To this end, trajectories are generated as well as evaluated in a novel twofold approach specifically tailored to the scenario. Finally, the position uncertainty can be extracted from the graph structure and subsequently be used to dynamically control the trade-off between localization and exploration. In a numerical evaluation, it is shown that the localization performance of the Graph SLAM approach to navigation in the vicinity of asteroids is generally high. Furthermore, the active perception strategy is able to trade-off between localization performance and the degree of exploration of the asteroid. Finally, when the latter process is dynamically controlled, based on the current localization uncertainty, a joint improvement of localization as well as exploration performance can be achieved. In addition, this thesis comprises an excursion into active sensorimotor object recognition. A sensorimotor feature is derived from biological principles of the human perceptual system. This feature is then employed in different probabilistic classification schemes. Furthermore, it enables the implementation of an active perception strategy, which can be thought of as a feature selection process in a classification scheme. It is shown that those strategies might be driven by top-down factors, i.e., based on previously learned information, or by bottom-up factors, i.e., based on saliency detected in the currently considered data. Evaluations are conducted based on real data acquired by a camera mounted on a robotic arm as well as on datasets. It is shown that the integrated representation of perception and action fosters classification performance and that the application of an active perception strategy accelerates the classification process

Distributed, Scalable And Resilient Information Acquisition For Multi-Robot Teams

Advances in robotic mobility and sensing technology have the potential to provide newcapabilities in a wide variety of information acquisition problems including environmental monitoring, structure inspection, localization and mapping of unknown environments, and search and rescue, amongst many others. In particular, teams composed of multiple robots have shown great potential in solving these problems, though it is challenging to design efficient algorithms that are distributed and scale well, and even more complex in hazardous or challenging environments. The purpose of this dissertation is to provide novel algorithms to the capabilities of multi-robot teams to gather information which are distributed, scalable, and resilient. The first part of the dissertation introduces the single-robot information acquisition problem, and focuses on algorithms that may be used for individual robots to plan their own trajectories. The methods presented here are search-based, meaning that an individual robot has a finite set of actions and is seeking to efficiently build a search tree over a known planning horizon. The first method presented details how to use the concept of algebraic redundancy and closeness to achieve a smooth trade-off of completeness in the exploration process, as an anytime planning algorithm. Next we show how a single robot can compute an admissible and consistent heuristic which guides the search towards the most informative regions of the state space, using the classic A* planning algorithm, drastically improving the search efficiency. The next chapter of the dissertation focuses on how to build on the single robot planning algorithms to create efficient algorithms for multi-robot teams, which operate in a distributed manner and scalable manner. The first method presented is coordinate descent, 5 otherwise known in the literature as sequential greedy assignment. This algorithm is implemented in a multi-robot target tracking hardware experiment. Next, we formulate an energy-aware multi-robot information acquisition problem, which allows for heterogeneity and captures trade-offs between information and energy expenditure. However, this results in a non-monotone objective function. Therefore we propose a new algorithm based on distributed local search, which achieves performance guarantees through a diminishing returns property known as submodularity. The final chapter focuses on hazardous or failure prone environments that necessitate resilience to a fixed number of failures in the multi-robot team. We provide a definition of resilience, and formulate a resilient information acquisition problem. We then propose the first algorithm that solves this problem through an online application of robust trajectory planning, and provide theoretical guarantees on its performance. We then present three unique applications of the resilient multi-robot information acquisition framework, including target tracking, occupancy grid mapping, and persistent surveillance which demonstrate the efficacy of our approach