Graph Reinforcement Learning Application to Co-operative Decision-Making in Mixed Autonomy Traffic: Framework, Survey, and Challenges

Proper functioning of connected and automated vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaboration decision-making for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) has been widely used in solving decision-making problems. However, the existing DRL-based methods have been mainly focused on solving the decision-making of a single CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot accurately represent the mutual effects of vehicles and model dynamic traffic environments. To address these shortcomings, this article proposes a graph reinforcement learning (GRL) approach for multi-agent decision-making of CAVs in mixed autonomy traffic. First, a generic and modular GRL framework is designed. Then, a systematic review of DRL and GRL methods is presented, focusing on the problems addressed in recent research. Moreover, a comparative study on different GRL methods is further proposed based on the designed framework to verify the effectiveness of GRL methods. Results show that the GRL methods can well optimize the performance of multi-agent decision-making for CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges and future research directions are summarized. This study can provide a valuable research reference for solving the multi-agent decision-making problems of CAVs in mixed autonomy traffic and can promote the implementation of GRL-based methods into intelligent transportation systems. The source code of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.Comment: 22 pages, 7 figures, 10 tables. Currently under review at IEEE Transactions on Intelligent Transportation System

Correct-by-Construction Tactical Planners for Automated Cars

One goal of developing automated cars is to completely free people from driving tasks. Automated cars that require no human driver need to handle all traffic situations that a human driver is expected to handle, and possibly more. Although human drivers cause a lot of traffic accidents, they still have a very low accident and failure rate that automated systems must match.Tactical planners are responsible for making discrete decisions during the coming seconds or minute. As with all subsystems in an automated car, these planners need to be supported with a credible and convincing argument of their correctness. The planners\u27 decisions affect the environment and the planners need to interact with other road users in a feedback loop, so the correctness of the planners depend on their behavior in relation to other drivers and the environment over time. One possibility to ascertain their correctness is to deploy the planners in real traffic. To be sufficiently certain that a tactical planner is safe by that methods, it needs to be tested on 255 million miles without having an accident.Formal methods can, in contrast to testing, mathematically prove that the requirements are fulfilled. Hence, they are a promising alternative for making credible arguments of tactical planners\u27 correctness. The topic of this thesis is how formal methods can be used in the automotive industry to design safe tactical planners. What is interesting is both how automotive systems should be modeled in formal frameworks, and how formal methods can be used practically within the automotive development process.The main findings of this thesis are that it is natural to express desired properties of tactical planners in formal languages and use formal methods to prove their correctness. Model Checking, Reactive Synthesis, and Supervisory Control Theory have been used in the design and development process of tactical planners, and all three methods have their benefits, depending on the application.Formal synthesis is an especially interesting class of formal methods because they can automatically generate a planner based on requirements and models. Formal synthesis removes the need to manually develop and implement the planner, so the development efforts can be directed to formalizing good requirements on the planner and good assumptions on the environment. However, formal synthesis has two limitations: the resulting planner is a black box that is difficult to inspect, and it is difficult to find a level of abstraction that allows detailed requirements and generic planners

Learning faster to perform autonomous lane changes by constructing maneuvers from shielded semantic actions

This paper introduces a new method to solve tactical decision making problems for highway lane changes. In the system design, reference sets for low level controllers are employed to formulate semantic meaningful actions used by reinforcement learning algorithm. Safety is ensured by preemptively shielding the Markov decision process (MDP) from unsafe actions. This frees the agent to focus on learning how to interact efficiently with the surrounding traffic. By introducing human demonstration with supervised loss as better exploration strategy, the learning process and initial performance are boosted further.\ua0\ua9 2019 IEEE

Real-time sensor data development for smart truck drivetrains

Heavy articulated transport vehicles have a poor reputation associated with dramatic road accidents with frequent fatalities for those in automobiles. The result of this work is a formal data flow structure to enhance real-time decision-making in complex mechanical systems to increase performance capability and responsiveness to human commands. This structure recognizes the multiple layers of highly non-linear mechanical components (actuators, wheel tire & ground surfaces, controllers, power supplies, human/machine interfaces, etc.) that must operate in unison (i.e., reduce conflicts) in real-time (in milli-seconds) to enhance operator (driver) control to maximize human choice. This work contains a discussion on dependable sensor data is vital in complex systems that rely on a suite of sensors for both control as well as condition monitoring purposes as well as discussion on real-time energy distribution analysis in high momentum mechanical systems. The focus will be on tractor trucks of class 7 & 8 that are outfitted with an array of low-cost redundant sensors leveraging advances in intelligent robotic systems. This work details many topics including: Most relevant sensor types and their technologies, Designing, implementing, and maintaining a multi-sensor system using feasible industry standards, Sensor signal integrity and data flow processing for decision making, Asynchronous data flow methods for operating decision making schemes in real-time, Multiple applications to enhance tractor trucks systems with multi-sensor systems for real-time decision making.Mechanical Engineerin

Autonomous mobility scooters as assistive tools for the elderly

The aim of this research is to investigate the development of an autonomous navigation system that could be used as an assistive tool for elderly and disabled people in their activities of daily living. The navigation environment is an urban environment and the platform is a Mobility Scooter (MoS). To achieve this aim, a differentially steered MoS was modifed to receive motion commands from a computer and outfitted with onboard sensors that included a Global Positioning System (GPS) receiver and two 2D planar laser range sensors. Perception methods were developed to detect the presence of an outdoor pedestrian walkway. These methods achieved this by processing the range data produced by the laser sensors to identify features that are typically found around walkways like curbs, low vegetation, walls and barriers. A method that utilises GPS localisation information to plan and navigate a route in an outdoor urban environment was also developed. Extensive experimental work was conducted to test the accuracy, repeatability and usefulness of the sensory devices. The developed perception methodologies were evaluated in real world environments while the navigation algorithms were predominantly tested in virtual environments. A navigation system that plans a route in an urban environment and follows it using behaviours arranged in a hierarchy is presented and shown to have the ability to safely navigate an MoS along an outdoor pedestrian path

Aerial Vehicles

This book contains 35 chapters written by experts in developing techniques for making aerial vehicles more intelligent, more reliable, more flexible in use, and safer in operation.It will also serve as an inspiration for further improvement of the design and application of aeral vehicles. The advanced techniques and research described here may also be applicable to other high-tech areas such as robotics, avionics, vetronics, and space

The Sixth Annual Workshop on Space Operations Applications and Research (SOAR 1992)

This document contains papers presented at the Space Operations, Applications, and Research Symposium (SOAR) hosted by the U.S. Air Force (USAF) on 4-6 Aug. 1992 and held at the JSC Gilruth Recreation Center. The symposium was cosponsored by the Air Force Material Command and by NASA/JSC. Key technical areas covered during the symposium were robotic and telepresence, automation and intelligent systems, human factors, life sciences, and space maintenance and servicing. The SOAR differed from most other conferences in that it was concerned with Government-sponsored research and development relevant to aerospace operations. The symposium's proceedings include papers covering various disciplines presented by experts from NASA, the USAF, universities, and industry

Fifth Conference on Artificial Intelligence for Space Applications

The Fifth Conference on Artificial Intelligence for Space Applications brings together diverse technical and scientific work in order to help those who employ AI methods in space applications to identify common goals and to address issues of general interest in the AI community. Topics include the following: automation for Space Station; intelligent control, testing, and fault diagnosis; robotics and vision; planning and scheduling; simulation, modeling, and tutoring; development tools and automatic programming; knowledge representation and acquisition; and knowledge base/data base integration

Technology 2003: The Fourth National Technology Transfer Conference and Exposition, volume 2

Proceedings from symposia of the Technology 2003 Conference and Exposition, Dec. 7-9, 1993, Anaheim, CA, are presented. Volume 2 features papers on artificial intelligence, CAD&E, computer hardware, computer software, information management, photonics, robotics, test and measurement, video and imaging, and virtual reality/simulation