Surveillance Planning against Smart Insurgents in Complex Terrain

This study is concerned with finding a way to solve a surveillance system allocation problem based on the need to consider intelligent insurgency that takes place in a complex geographical environment. Although this effort can be generalized to other situations, it is particularly geared towards protecting military outposts in foreign lands. The technological assets that are assumed available include stare-devices, such as tower-cameras and aerostats, as well as manned and unmanned aerial systems. Since acquiring these assets depends on the ability to control and monitor them on the target terrain, their operations on the geo-location of interest ought to be evaluated. Such an assessment has to also consider the risks associated with the environmental advantages that are accessible to a smart adversary. Failure to consider these aspects might render the forces vulnerable to surprise attacks. The problem of this study is formulated as follows: given a complex terrain and a smart adversary, what types of surveillance systems, and how many entities of each kind, does a military outpost need to adequately monitor its surrounding environment? To answer this question, an analytical framework is developed and structured as a series of problems that are solved in a comprehensive and realistic fashion. This includes digitizing the terrain into a grid of cell objects, identifying high-risk spots, generating flight tours, and assigning the appropriate surveillance system to the right route or area. Optimization tools are employed to empower the framework in enforcing constraints--such as fuel/battery endurance, flying assets at adequate altitudes, and respecting the climbing/diving rate limits of the aerial vehicles--and optimizing certain mission objectives--e.g. revisiting critical regions in a timely manner, minimizing manning requirements, and maximizing sensor-captured image quality. The framework is embedded in a software application that supports a friendly user interface, which includes the visualization of maps, tours, and related statistics. The final product is expected to support designing surveillance plans for remote military outposts and making critical decisions in a more reliable manner

Game-Theoretic Safety Assurance for Human-Centered Robotic Systems

In order for autonomous systems like robots, drones, and self-driving cars to be reliably introduced into our society, they must have the ability to actively account for safety during their operation. While safety analysis has traditionally been conducted offline for controlled environments like cages on factory floors, the much higher complexity of open, human-populated spaces like our homes, cities, and roads makes it unviable to rely on common design-time assumptions, since these may be violated once the system is deployed. Instead, the next generation of robotic technologies will need to reason about safety online, constructing high-confidence assurances informed by ongoing observations of the environment and other agents, in spite of models of them being necessarily fallible.This dissertation aims to lay down the necessary foundations to enable autonomous systems to ensure their own safety in complex, changing, and uncertain environments, by explicitly reasoning about the gap between their models and the real world. It first introduces a suite of novel robust optimal control formulations and algorithmic tools that permit tractable safety analysis in time-varying, multi-agent systems, as well as safe real-time robotic navigation in partially unknown environments; these approaches are demonstrated on large-scale unmanned air traffic simulation and physical quadrotor platforms. After this, it draws on Bayesian machine learning methods to translate model-based guarantees into high-confidence assurances, monitoring the reliability of predictive models in light of changing evidence about the physical system and surrounding agents. This principle is first applied to a general safety framework allowing the use of learning-based control (e.g. reinforcement learning) for safety-critical robotic systems such as drones, and then combined with insights from cognitive science and dynamic game theory to enable safe human-centered navigation and interaction; these techniques are showcased on physical quadrotors—flying in unmodeled wind and among human pedestrians—and simulated highway driving. The dissertation ends with a discussion of challenges and opportunities ahead, including the bridging of safety analysis and reinforcement learning and the need to ``close the loop'' around learning and adaptation in order to deploy increasingly advanced autonomous systems with confidence

Behavior planning for automated highway driving

This work deals with certain components of an automated driving system for highways, focusing on lane change behavior planning. It presents a variety of algorithms of a modular system aiming at safe and comfortable driving. A major contribution of this work is a method for analyzing traffic scenes in a spatio-temporal, curvilinear coordinate system. The results of this analysis are used in a further step to generate lane change trajectories. A total of three approaches with increasing levels of complexity and capabilities are compared. The most advanced approach formulates the problem as a linear-quadratic cooperative game and accounts for the inherently uncertain and multimodal nature of trajectory predictions for surrounding road users. Evaluations on real data show that the developed algorithms can be integrated into current generation automated driving software systems fulfilling runtime constraints

Representing the Unknown - Impact of Uncertainty on the Interaction between Decision Making and Trajectory Generation

Even though motion planning for automated vehicles has been extensively discussed for more than two decades, it is still a highly active field of research with a variety of different approaches having been published in the recent years. When considering the market introduction of SAE Level 3+ vehicles, the topic of motion planning will most likely be subject to even more detailed discussions between safety and user acceptance. This paper shall discuss parameters of the motion planning problem and requirements to an environment model. The focus is put on the representation of different types of uncertainty at the example of sensor occlusion, arguing the importance of a well-defined interface between decision making and trajectory generation

Belief State Planning for Autonomous Driving: Planning with Interaction, Uncertain Prediction and Uncertain Perception

This thesis presents a behavior planning algorithm for automated driving in urban environments with an uncertain and dynamic nature. The uncertainty in the environment arises by the fact that the intentions as well as the future trajectories of the surrounding drivers cannot be measured directly but can only be estimated in a probabilistic fashion. Even the perception of objects is uncertain due to sensor noise or possible occlusions. When driving in such environments, the autonomous car must predict the behavior of the other drivers and plan safe, comfortable and legal trajectories. Planning such trajectories requires robust decision making when several high-level options are available for the autonomous car. Current planning algorithms for automated driving split the problem into different subproblems, ranging from discrete, high-level decision making to prediction and continuous trajectory planning. This separation of one problem into several subproblems, combined with rule-based decision making, leads to sub-optimal behavior. This thesis presents a global, closed-loop formulation for the motion planning problem which intertwines action selection and corresponding prediction of the other agents in one optimization problem. The global formulation allows the planning algorithm to make the decision for certain high-level options implicitly. Furthermore, the closed-loop manner of the algorithm optimizes the solution for various, future scenarios concerning the future behavior of the other agents. Formulating prediction and planning as an intertwined problem allows for modeling interaction, i.e. the future reaction of the other drivers to the behavior of the autonomous car. The problem is modeled as a partially observable Markov decision process (POMDP) with a discrete action and a continuous state and observation space. The solution to the POMDP is a policy over belief states, which contains different reactive plans for possible future scenarios. Surrounding drivers are modeled with interactive, probabilistic agent models to account for their prediction uncertainty. The field of view of the autonomous car is simulated ahead over the whole planning horizon during the optimization of the policy. Simulating the possible, corresponding, future observations allows the algorithm to select actions that actively reduce the uncertainty of the world state. Depending on the scenario, the behavior of the autonomous car is optimized in (combined lateral and) longitudinal direction. The algorithm is formulated in a generic way and solved online, which allows for applying the algorithm on various road layouts and scenarios. While such a generic problem formulation is intractable to solve exactly, this thesis demonstrates how a sufficiently good approximation to the optimal policy can be found online. The problem is solved by combining state of the art Monte Carlo tree search algorithms with near-optimal, domain specific roll-outs. The algorithm is evaluated in scenarios such as the crossing of intersections under unknown intentions of other crossing vehicles, interactive lane changes in narrow gaps and decision making at intersections with large occluded areas. It is shown that the behavior of the closed-loop planner is less conservative than comparable open-loop planners. More precisely, it is even demonstrated that the policy enables the autonomous car to drive in a similar way as an omniscient planner with full knowledge of the scene. It is also demonstrated how the autonomous car executes actions to actively gather more information about the surrounding and to reduce the uncertainty of its belief state