Safe Planning in Dynamic Environments using Conformal Prediction

We propose a framework for planning in unknown dynamic environments with probabilistic safety guarantees using conformal prediction. Particularly, we design a model predictive controller (MPC) that uses i) trajectory predictions of the dynamic environment, and ii) prediction regions quantifying the uncertainty of the predictions. To obtain prediction regions, we use conformal prediction, a statistical tool for uncertainty quantification, that requires availability of offline trajectory data - a reasonable assumption in many applications such as autonomous driving. The prediction regions are valid, i.e., they hold with a user-defined probability, so that the MPC is provably safe. We illustrate the results in the self-driving car simulator CARLA at a pedestrian-filled intersection. The strength of our approach is compatibility with state of the art trajectory predictors, e.g., RNNs and LSTMs, while making no assumptions on the underlying trajectory-generating distribution. To the best of our knowledge, these are the first results that provide valid safety guarantees in such a setting

Interactive Prediction and Planning for Autonomous Driving: from Algorithms to Fundamental Aspects

Inevitably, autonomous vehicles need to interact with other road participants in a variety of highly complex or critical driving scenarios. It is still an extremely challenging task even for the forefront companies or institutes to enable autonomous vehicles to interactively predict the behavior of others, and plan safe and high-quality motions accordingly. The major obstacles are not just originated from prediction and planning algorithms with insufficient performances. Several fundamental problems in the fields of interactive prediction and planning still remain open, such as formulation, representation and evaluation of interactive prediction methods, motion dataset with densely interactive driving behavior, as well as interface of interactive prediction and planning algorithms. The aforementioned fundamental aspects of interactive prediction and planning are addressed in this dissertation along with various kinds of algorithms. First, generic environmental representation for various scenarios with topological decomposition is constructed, and a corresponding planning algorithm is designed by combining graph search and optimization. Hard constraints in optimization-based planners are also incorporated into the training loss of imitation learning so that the policy net can generate safe and feasible motions in highly constrained scenarios. Unified problem formulation and motion representation are designed for different paradigms of interactive predictors such as planning-based prediction (inverse reinforcement learning), as well as probabilistic graphical models (hidden Markov model) and deep neural networks (mixture density network), which are utilized for the prediction/planning interface design and prediction benchmark. A framework combing decision network and graph-search/optimization/sample-based planner is proposed to achieve a driving strategy which is defensive to potential violations of others, but not overly conservatively to threats of low probabilities. Such driving strategy is achieved via experiments based on the aforementioned interactive prediction and planning algorithms with proper interface designed. These predictors are also evaluated from closed loop perspective considering planning fatality when using the prediction results instead of pure data approximation metrics. Finally, INTERACTION (INTERnational, Adversarial and Cooperative moTION) dataset with highly interactive driving scenarios and behavior from international locations is constructed with interaction density metric defined to compare different datasets. The dataset has been utilized for various behavior-related research areas such as prediction, planning, imitation learning and behavior modeling, and is inspiring new research fields such as representation learning, interaction extraction and scenario generation

Autonomous terminal area operations for unmanned aerial systems

After many years of successful operation in military domains, Unmanned Aerial Systems (UASs) are generating significant interest amongst civilian operators in sectors such as law enforcement, search and rescue, aerial photography and mapping. To maximise the benefits brought by UASs to sectors such as these, a high level of autonomy is desirable to reduce the need for highly skilled operators. Highly autonomous UASs require a high level of situation awareness in order to make appropriate decisions. This is of particular importance to civilian UASs where transparency and equivalence of operation to current manned aircraft is a requirement, particularly in the terminal area immediately surrounding an airfield. This thesis presents an artificial situation awareness system for an autonomous UAS capable of comprehending both the current continuous and discrete states of traffic vehicles. This estimate forms the basis of the projection element of situation awareness, predicting the future states of traffic. Projection is subject to a large degree of uncertainty in both continuous state variables and in the execution of intent information by the pilot. Both of these sources of uncertainty are captured to fully quantify the future positions of traffic. Based upon the projection of future traffic positions a self separation system is designed which allows an UAS to quantify its separation to traffic vehicles up to some future time and manoeuvre appropriately to minimise the potential for conflict. A high fidelity simulation environment has been developed to test the performance of the artificial situation awareness and self separation system. The system has demonstrated good performance under all situations, with an equivalent level of safety to that of a human pilot

Architecture and Information Requirements to Assess and Predict Flight Safety Risks During Highly Autonomous Urban Flight Operations

As aviation adopts new and increasingly complex operational paradigms, vehicle types, and technologies to broaden airspace capability and efficiency, maintaining a safe system will require recognition and timely mitigation of new safety issues as they emerge and before significant consequences occur. A shift toward a more predictive risk mitigation capability becomes critical to meet this challenge. In-time safety assurance comprises monitoring, assessment, and mitigation functions that proactively reduce risk in complex operational environments where the interplay of hazards may not be known (and therefore not accounted for) during design. These functions can also help to understand and predict emergent effects caused by the increased use of automation or autonomous functions that may exhibit unexpected non-deterministic behaviors. The envisioned monitoring and assessment functions can look for precursors, anomalies, and trends (PATs) by applying model-based and data-driven methods. Outputs would then drive downstream mitigation(s) if needed to reduce risk. These mitigations may be accomplished using traditional design revision processes or via operational (and sometimes automated) mechanisms. The latter refers to the in-time aspect of the system concept. This report comprises architecture and information requirements and considerations toward enabling such a capability within the domain of low altitude highly autonomous urban flight operations. This domain may span, for example, public-use surveillance missions flown by small unmanned aircraft (e.g., infrastructure inspection, facility management, emergency response, law enforcement, and/or security) to transportation missions flown by larger aircraft that may carry passengers or deliver products. Caveat: Any stated requirements in this report should be considered initial requirements that are intended to drive research and development (R&D). These initial requirements are likely to evolve based on R&D findings, refinement of operational concepts, industry advances, and new industry or regulatory policies or standards related to safety assurance

Conformal Prediction for STL Runtime Verification

We are interested in predicting failures of cyber-physical systems during their operation. Particularly, we consider stochastic systems and signal temporal logic specifications, and we want to calculate the probability that the current system trajectory violates the specification. The paper presents two predictive runtime verification algorithms that predict future system states from the current observed system trajectory. As these predictions may not be accurate, we construct prediction regions that quantify prediction uncertainty by using conformal prediction, a statistical tool for uncertainty quantification. Our first algorithm directly constructs a prediction region for the satisfaction measure of the specification so that we can predict specification violations with a desired confidence. The second algorithm constructs prediction regions for future system states first, and uses these to obtain a prediction region for the satisfaction measure. To the best of our knowledge, these are the first formal guarantees for a predictive runtime verification algorithm that applies to widely used trajectory predictors such as RNNs and LSTMs, while being computationally simple and making no assumptions on the underlying distribution. We present numerical experiments of an F-16 aircraft and a self-driving car

2020 NASA Technology Taxonomy

This document is an update (new photos used) of the PDF version of the 2020 NASA Technology Taxonomy that will be available to download on the OCT Public Website. The updated 2020 NASA Technology Taxonomy, or "technology dictionary", uses a technology discipline based approach that realigns like-technologies independent of their application within the NASA mission portfolio. This tool is meant to serve as a common technology discipline-based communication tool across the agency and with its partners in other government agencies, academia, industry, and across the world

Human Behavior Modeling and Human Behavior-aware Control of Automated Vehicles for Trustworthy Navigation

First and foremost, I would like to thank my advisor, Professor Dawn Tilbury, for her constant guidance and encouragement. She has been extremely helpful in developing my technical, research, and personal skills and immensely supportive of my ideas and endeavors throughout graduate school. She has been an excellent mentor and has always been there in my time of need, encouraging and boosting my confidence when I needed them the most. I would like to specially thank my committee members and collaborators, Professors Lionel Robert and Jessie Yang, for their support and encouragement, right from the start of my graduate program. The multi-disciplinary nature of the research initiated by these three Professors is what first drew me towards pursuing a Ph.D. I would also like to thank my other committee members Professors Ilya Kolmanovsky and Ram Vasudevan, for providing their support and feedback that improved the dissertation. I would like to thank the Department of Mechanical Engineering, Rackham Graduate School, and the University of Michigan for giving me the opportunity to pursue the doctoral degree and providing financial support during my time at the university. In addition, I would like to thank the Toyota Research Institute and the Automotive Research Center for providing financial assistance. I really appreciate the support I received from the MAVRIC lab members. The multi-disciplinary culture and environment that the Professors have fostered in the MAVRIC lab have deeply broadened my perspectives. Specically, I would like to thank Hebert Azevedo-Sa. He is usually the first person I discuss my ideas with and has been an excellent critique. I would also like to thank Connor Esterwood, Na Du, Qiaoning Zhang, and Huajing Zhao for the numerous discussions and help with my user studies; especially Connor, who took on a variety of roles to help with my user study|from an engineer to a tailor, to even a hidden driver. Outside of the University of Michigan, I would like to thank my undergraduate advisor, Professor Madhu M., and my internship advisor at the Indian Institute of Technology-Madras, Professor Saravanan Gurunathan. They encouraged me to pursue research and provided me with the necessary opportunities. A special thanks to Sajaysurya Ganesh, a close friend, and collaborator in my early research projects, with who I discuss ideas even now. Last but not least, I would like to thank my family and friends for supporting me during the past several years. My friends at Ann Arbor made life away from home much easier; they are like my second family. A long list of people from my Master's and Ph.D. programs at the University of Michigan has played an essential role in my graduate experience. Still, I would like to especially thank Sandipp Krishnan Ravi, Subramaniam Balakrishna, Rahasudha Kannan, and Paavai Pari for all their love and support. I will fondly remember my time at the University of Michigan and in Ann Arbor because of all of the people I encountered, the friends I made, and the experiences I had. My parents, wife, and extended family have all been incredibly supportive of the pursuit of my degree, and I am eternally grateful for their love and guidance.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169640/1/jskumaar_1.pd

Review of graph-based hazardous event detection methods for autonomous driving systems

Automated and autonomous vehicles are often required to operate in complex road environments with potential hazards that may lead to hazardous events causing injury or even death. Therefore, a reliable autonomous hazardous event detection system is a key enabler for highly autonomous vehicles (e.g., Level 4 and 5 autonomous vehicles) to operate without human supervision for significant periods of time. One promising solution to the problem is the use of graph-based methods that are powerful tools for relational reasoning. Using graphs to organise heterogeneous knowledge about the operational environment, link scene entities (e.g., road users, static objects, traffic rules) and describe how they affect each other. Due to a growing interest and opportunity presented by graph-based methods for autonomous hazardous event detection, this paper provides a comprehensive review of the state-of-the-art graph-based methods that we categorise as rule-based, probabilistic, and machine learning-driven. Additionally, we present an in-depth overview of the available datasets to facilitate hazardous event training and evaluation metrics to assess model performance. In doing so, we aim to provide a thorough overview and insight into the key research opportunities and open challenges