Cooperative Path-Planning for Multi-Vehicle Systems

In this paper, we propose a collision avoidance algorithm for multi-vehicle systems, which is a common problem in many areas, including navigation and robotics. In dynamic environments, vehicles may become involved in potential collisions with each other, particularly when the vehicle density is high and the direction of travel is unrestricted. Cooperatively planning vehicle movement can effectively reduce and fairly distribute the detour inconvenience before subsequently returning vehicles to their intended paths. We present a novel method of cooperative path planning for multi-vehicle systems based on reinforcement learning to address this problem as a decision process. A dynamic system is described as a multi-dimensional space formed by vectors as states to represent all participating vehicles’ position and orientation, whilst considering the kinematic constraints of the vehicles. Actions are defined for the system to transit from one state to another. In order to select appropriate actions whilst satisfying the constraints of path smoothness, constant speed and complying with a minimum distance between vehicles, an approximate value function is iteratively developed to indicate the desirability of every state-action pair from the continuous state space and action space. The proposed scheme comprises two phases. The convergence of the value function takes place in the former learning phase, and it is then used as a path planning guideline in the subsequent action phase. This paper summarizes the concept and methodologies used to implement this online cooperative collision avoidance algorithm and presents results and analysis regarding how this cooperative scheme improves upon two baseline schemes where vehicles make movement decisions independently

Clustering-based algorithms for multi-vehicle task assignment in a time-invariant drift field

This paper studies the multi-vehicle task assignment problem where several dispersed vehicles need to visit a set of target locations in a time-invariant drift field while trying to minimize the total travel time. Using optimal control theory, we first design a path planning algorithm to minimize the time for each vehicle to travel between two given locations in the drift field. The path planning algorithm provides the cost matrix for the target assignment, and generates routes once the target locations are assigned to a vehicle. Then, we propose several clustering strategies to assign the targets, and we use two metrics to determine the visiting sequence of the targets clustered to each vehicle. Mainly used to specify the minimum time for a vehicle to travel between any two target locations, the cost matrix is obtained using the path planning algorithm, and is in general asymmetric due to time-invariant currents of the drift field. We show that one of the clustering strategies can obtain a min-cost arborescence of the asymmetric target vehicle graph where the weight of a directed edge between two vertices is the minimum travel time from one vertex to the other respecting the orientation. Using tools from graph theory, a lower bound on the optimal solution is found, which can be used to measure the proximity of a solution from the optimal. Furthermore, by integrating the target clustering strategies with the target visiting metrics, we obtain several task assignment algorithms. Among them, two algorithms guarantee that all the target locations will be visited within a computable maximal travel time, which is at most twice of the optimal when the cost matrix is symmetric. Finally, numerical simulations show that the algorithms can quickly lead to a solution that is close to the optimal

Deep Reinforcement Learning Based Joint 3D Navigation and Phase Shift Control for Mobile Internet of Vehicles Assisted by RIS-equipped UAVs

Unmanned aerial vehicles (UAVs) are utilized to improve the performance of wireless communication networks (WCNs), notably, in the context of Internet-of-things (IoT). However, the application of UAVs, as active aerial base stations (BSs)/relays, is questionable in the fifth-generation (5G) WCNs with quasi-optic millimeter wave (mmWave) and beyond in 6G (visible light) WCNs. Because path loss is high in 5G/6G networks that attenuate, even, the line-of-sight (LoS) communicating signals propagated by UAVs. Besides, the limited energy/size/weight of UAVs makes it cost-deficient to design aerial multi-input/output BSs for active beamforming to strengthen the signals. Equipping UAVs with the reconfigurable intelligent surface (RIS), a passive component, can help to address the problems with UAV-assisted communication in 5G and optical 6G networks. We propose adopting the RIS-equipped UAV (RISeUAV) to provide aerial LoS service and facilitate communication for mobile Internet-of-vehicles (IoVs) in an obstructed dense urban area covered by 5G/6G. RISeUAV-aided wireless communication facilitates vehicle-to-vehicle/everything communication for IoVs for updating IoT information required for sensor fusion and autonomous driving. However, autonomous navigation of RISeUAV for this purpose is a multilateral problem and is computationally challenging for being optimally implemented in real-time. We intelligently automated RISeUAV navigation using deep reinforcement learning to address the optimality and time complexity issues. Simulation results show the effectiveness of the method

MULTI-VEHICLE ROUTE PLANNING FOR CENTRALIZED AND DECENTRALIZED SYSTEMS

Multi-vehicle route planning is the problem of determining routes for a set of vehicles to visit a set of locations of interest. In this thesis, we describe a study of a classical multi-vehicle route planning problem which compared existing solutions methods on min-sum (minimizing total distance traveled) and min-max (minimizing maximum distance traveled) cost objectives. We then extended the work in this study by adapting approaches tested to generate robust solutions to a failure-robust multi vehicle route planning problem in which a potential vehicle failure may require modifying the solution, which could increase costs. Additionally, we considered a decentralized extension to the multi-vehicle route planning problem, also known as the decentralized task allocation problem. The results of a computational study show that our novel genetic algorithm generated better solutions than existing approaches on larger instances with high communication quality

Distributed Task Allocation and Task Sequencing for Robots with Motion Constraints

This thesis considers two routing and scheduling problems. The first problem is task allocation and sequencing for multiple robots with differential motion constraints. Each task is defined as visiting a point in a subset of the robot configuration space -- this definition captures a variety of tasks including inspection and servicing, as well as one-in-a-set tasks. Our approach is to transform the problem into a multi-vehicle generalized traveling salesman problem (GTSP). We analyze the GTSP insertion methods presented in literature and we provide bounds on the performance of the three insertion mechanisms. We then develop a combinatorial-auction-based distributed implementation of the allocation and sequencing algorithm. The number of the bids in a combinatorial auction, a crucial factor in the runtime, is shown to be linear in the size of the tasks. Finally, we present extensive benchmarking results to demonstrate the improvement over existing distributed task allocation methods. In the second part of this thesis, we address the problem of computing optimal paths through three consecutive points for the curvature-constrained forward moving Dubins vehicle. Given initial and final configurations of the Dubins vehicle and a midpoint with an unconstrained heading, the objective is to compute the midpoint heading that minimizes the total Dubins path length. We provide a novel geometrical analysis of the optimal path and establish new properties of the optimal Dubins' path through three points. We then show how our method can be used to quickly refine Dubins TSP tours produced using state-of-the-art techniques. We also provide extensive simulation results showing the improvement of the proposed approach in both runtime and solution quality over the conventional method of uniform discretization of the heading at the mid-point, followed by solving the minimum Dubins path for each discrete heading

Receding Horizon based Cooperative Vehicle Control with Optimal Task Allocation

The problem of cooperative multi-target interception in an uncertain environment is investigated in this thesis. The targets arrive in the mission space sequentially at a priori unknown time instants and a priori unknown locations, and then move on a priori unknown trajectories. A group of vehicles with known dynamics are employed to visit the targets as quickly and efficiently as possible. To this end, a time-discounting reward is defined for each target which can be collected only if one of the vehicles visits that target. A cooperative receding horizon scheme is designed, which predicts the future positions of the targets and maximizes the estimate of the expected total collectible rewards, accordingly. The problem is initially investigated for the case when there are a finite number of targets arriving in the mission space sequentially. It is shown that the number of targets that are not visited by any vehicle in the mission space will be sufficiently small if the targets arrive sufficiently infrequently. The problem is then generalized to the case of infinite number of targets and a finite-time convergence analysis is also presented. A more practical case where the vehicles have limited sensing and communication ranges is also investigated using a game-theoretic approach. The problem is then solved for the case when a cluster of vehicles is required to visit each target. Simulations confirm the efficacy of the proposed strategies

Market_based Framework for Mobile Surveillance Systems

The active surveillance of public and private sites is increasingly becoming a very important and critical issue. It is therefore, imperative to develop mobile surveillance systems to protect these sites. Modern surveillance systems encompass spatially distributed mobile and static sensors in order to provide effective monitoring of persistent and transient objects and events in a given Area Of Interest (AOI). The realization of the potential of mobile surveillance requires the solution of different challenging problems such as task allocation, mobile sensor deployment, multisensor management, cooperative object detection and tracking, decentralized data fusion, and interoperability and accessibility of system nodes. This thesis proposes a market-based framework that can be used to handle different problems of mobile surveillance systems. Task allocation and cooperative target-tracking are studied using the proposed framework as two challenging problems of mobile surveillance systems. These challenges are addressed individually and collectively