Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

Circular formation control of fixed-wing UAVs with constant speeds

In this paper we propose an algorithm for stabilizing circular formations of fixed-wing UAVs with constant speeds. The algorithm is based on the idea of tracking circles with different radii in order to control the inter-vehicle phases with respect to a target circumference. We prove that the desired equilibrium is exponentially stable and thanks to the guidance vector field that guides the vehicles, the algorithm can be extended to other closed trajectories. One of the main advantages of this approach is that the algorithm guarantees the confinement of the team in a specific area, even when communications or sensing among vehicles are lost. We show the effectiveness of the algorithm with an actual formation flight of three aircraft. The algorithm is ready to use for the general public in the open-source Paparazzi autopilot.Comment: 6 pages, submitted to IROS 201

Safe Sequential Path Planning Under Disturbances and Imperfect Information

Multi-UAV systems are safety-critical, and guarantees must be made to ensure no unsafe configurations occur. Hamilton-Jacobi (HJ) reachability is ideal for analyzing such safety-critical systems; however, its direct application is limited to small-scale systems of no more than two vehicles due to an exponentially-scaling computational complexity. Previously, the sequential path planning (SPP) method, which assigns strict priorities to vehicles, was proposed; SPP allows multi-vehicle path planning to be done with a linearly-scaling computational complexity. However, the previous formulation assumed that there are no disturbances, and that every vehicle has perfect knowledge of higher-priority vehicles' positions. In this paper, we make SPP more practical by providing three different methods to account for disturbances in dynamics and imperfect knowledge of higher-priority vehicles' states. Each method has different assumptions about information sharing. We demonstrate our proposed methods in simulations.Comment: American Control Conference, 201

Calibration and Validation of A Shared space Model: A Case Study

Shared space is an innovative streetscape design that seeks minimum separation between vehicle traffic and pedestrians. Urban design is moving toward space sharing as a means of increasing the community texture of street surroundings. Its unique features aim to balance priorities and allow cars and pedestrians to coexist harmoniously without the need to dictate behavior. There is, however, a need for a simulation tool to model future shared space schemes and to help judge whether they might represent suitable alternatives to traditional street layouts. This paper builds on the authors’ previously published work in which a shared space microscopic mixed traffic model based on the social force model (SFM) was presented, calibrated, and evaluated with data from the shared space link typology of New Road in Brighton, United Kingdom. Here, the goal is to explore the transferability of the authors’ model to a similar shared space typology and investigate the effect of flow and ratio of traffic modes. Data recorded from the shared space scheme of Exhibition Road, London, were collected and analyzed. The flow and speed of cars and segregation between pedestrians and cars are greater on Exhibition Road than on New Road. The rule-based SFM for shared space modeling is calibrated and validated with the real data. On the basis of the results, it can be concluded that shared space schemes are context dependent and that factors such as the infrastructural design of the environment and the flow and speed of pedestrians and vehicles affect the willingness to share space

Traffic monitoring using image processing : a thesis presented in partial fulfillment of the requirements for the degree of Master of Engineering in Information and Telecommunications Engineering at Massey University, Palmerston North, New Zealand

Traffic monitoring involves the collection of data describing the characteristics of vehicles and their movements. Such data may be used for automatic tolls, congestion and incident detection, law enforcement, and road capacity planning etc. With the recent advances in Computer Vision technology, videos can be analysed automatically and relevant information can be extracted for particular applications. Automatic surveillance using video cameras with image processing technique is becoming a powerful and useful technology for traffic monitoring. In this research project, a video image processing system that has the potential to be developed for real-time application is developed for traffic monitoring including vehicle tracking, counting, and classification. A heuristic approach is applied in developing this system. The system is divided into several parts, and several different functional components have been built and tested using some traffic video sequences. Evaluations are carried out to show that this system is robust and can be developed towards real-time applications