Extended Kalman Filter based Resilient Formation Tracking Control of Multiple Unmanned Vehicles Via Game-Theoretical Reinforcement Learning

In This Paper, We Discuss the Resilient Formation Tracking Control Problem of Multiple Unmanned Vehicles (MUV). a Dynamic Leader-Follower Distributed Control Structure is Utilized to Optimize the Performance of the Formation Tracking. for the Follower of the MUV, the Leader is a Cooperative Unmanned Vehicle, and the Target of Formation Tracking is a Non-Cooperative Unmanned Vehicle with a Nonlinear Trajectory. Therefore, an Extended Kalman Filter (EKF) Observer is Designed to Estimate the State of the Target. Then the Leader of the MUV is Adjusted Dynamically According to the State of the Target. in Order to Describe the Interactions between the Follower and Dynamic Leader, a Stackelberg Game Model is Constructed to Handle the Hierarchical Decision Problems. at the Lower Layer, Each Follower Responds by Observing the Leader\u27s Strategy, and the Potential Game is Used to Prove a Nash Equilibrium among All Followers. at the Upper Layer, the Dynamic Leader Makes Decisions Depending on the Response of All Followers to Reaching the Stackelberg Equilibrium. Moreover, the Stackelberg-Nash Equilibrium of the Designed Game Theoretical Model is Proven. a Novel Reinforcement Learning-Based Algorithm is Designed to Achieve the Stackelberg-Nash Equilibrium of the Game. Finally, the Effectiveness of the Method is Verified by a Variety of Formation Tracking Simulation Experiments

Iterative learning control for multi-agent systems with impulsive consensus tracking

In this paper, we adopt D-type and PD-type learning laws with the initial state of iteration to achieve uniform tracking problem of multi-agent systems subjected to impulsive input. For the multi-agent system with impulse, we show that all agents are driven to achieve a given asymptotical consensus as the iteration number increases via the proposed learning laws if the virtual leader has a path to any follower agent. Finally, an example is illustrated to verify the effectiveness by tracking a continuous or piecewise continuous desired trajectory

Synchronous-clock range-angle relative acoustic navigation: a unified approach to multi-AUV localization, command, control, and coordination

© The Author(s), 2022. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Rypkema, N., Schmidt, H., & Fischell, E. Synchronous-clock range-angle relative acoustic navigation: a unified approach to multi-AUV localization, command, control, and coordination. Journal of Field Robotics, 2(1), (2022): 774–806, https://doi.org/10.55417/fr.2022026.This paper presents a scalable acoustic navigation approach for the unified command, control, and coordination of multiple autonomous underwater vehicles (AUVs). Existing multi-AUV operations typically achieve coordination manually by programming individual vehicles on the surface via radio communications, which becomes impractical with large vehicle numbers; or they require bi-directional intervehicle acoustic communications to achieve limited coordination when submerged, with limited scalability due to the physical properties of the acoustic channel. Our approach utilizes a single, periodically broadcasting beacon acting as a navigation reference for the group of AUVs, each of which carries a chip-scale atomic clock and fixed ultrashort baseline array of acoustic receivers. One-way travel-time from synchronized clocks and time-delays between signals received by each array element allow any number of vehicles within receive distance to determine range, angle, and thus determine their relative position to the beacon. The operator can command different vehicle behaviors by selecting between broadcast signals from a predetermined set, while coordination between AUVs is achieved without intervehicle communication by defining individual vehicle behaviors within the context of the group. Vehicle behaviors are designed within a beacon-centric moving frame of reference, allowing the operator to control the absolute position of the AUV group by repositioning the navigation beacon to survey the area of interest. Multiple deployments with a fleet of three miniature, low-cost SandShark AUVs performing closed-loop acoustic navigation in real-time provide experimental results validated against a secondary long-baseline positioning system, demonstrating the capabilities and robustness of our approach with real-world data.This work was partially supported by the Office of Naval Research, the Defense Advanced Research Projects Agency, Lincoln Laboratory, and the Reuben F. and Elizabeth B. Richards Endowed Funds at WHOI

Control law and state estimators design for multi-agent system with reduction of communications by event-triggered approach

A large amount of research work has been recently dedicated to the study of Multi-Agent System and cooperative control. Applications to mobile robots, like unmanned air vehicles (UAVs), satellites, or aircraft have been tackled to insure complex mission such as exploration or surveillance. However, cooperative tasking requires communication between agents, and for a large number of agents, the number of communication exchanges may lead to network saturation, increased delays or loss of transferred packets, from the interest in reducing them. In event-triggered strategy, a communication is broadcast when a condition, based on chosen parameters and some threshold, is fulfilled. The main difficulty consists in determining the communication triggering condition (CTC) that will ensure the completion of the task assigned to the MAS. In a distributed strategy, each agent maintains an estimate value of others agents state to replace missing information due to limited communication. This thesis focuses on the development of distributed control laws and estimators for multi-agent system to limit the number of communication by using event-triggered strategy in the presence of perturbation with two main topics, i.e. consensus and formation control. The first part addresses the problem of distributed event-triggered communications for consensus of a multi-agent system with both general linear dynamics and state perturbations. To decrease the amount of required communications, an accurate estimator of the agent states is introduced, coupled with an estimator of the estimation error, and adaptation of communication protocol. By taking into account the control input of the agents, the proposed estimator allows to obtain a consensus with fewer communications than those obtained by a reference method. The second part proposes a strategy to reduce the number of communications for displacement-based formation control while following a desired reference trajectory. Agent dynamics are described by Euler-Lagrange models with perturbations and uncertainties on the model parameters. Several estimator structures are proposed to rebuilt missing information. The proposed distributed communication triggering condition accounts for inter-agent displacements and the relative discrepancy between actual and estimated agent states. A single a priori trajectory has to be evaluated to follow the desired path. Effect of state perturbations on the formation and on the communications are analyzed. Finally, the proposed methods have been adapted to consider packet dropouts and communication delays. For both type

Event-triggering architectures for adaptive control of uncertain dynamical systems

In this dissertation, new approaches are presented for the design and implementation of networked adaptive control systems to reduce the wireless network utilization while guaranteeing system stability in the presence of system uncertainties. Specifically, the design and analysis of state feedback adaptive control systems over wireless networks using event-triggering control theory is first presented. The state feedback adaptive control results are then generalized to the output feedback case for dynamical systems with unmeasurable state vectors. This event-triggering approach is then adopted for large-scale uncertain dynamical systems. In particular, decentralized and distributed adaptive control methodologies are proposed with reduced wireless network utilization with stability guarantees. In addition, for systems in the absence of uncertainties, a new observer-free output feedback cooperative control architecture is developed. Specifically, the proposed architecture is predicated on a nonminimal state-space realization that generates an expanded set of states only using the filtered input and filtered output and their derivatives for each vehicle, without the need for designing an observer for each vehicle. Building on the results of this new observer-free output feedback cooperative control architecture, an event-triggering methodology is next proposed for the output feedback cooperative control to schedule the exchanged output measurements information between the agents in order to reduce wireless network utilization. Finally, the output feedback cooperative control architecture is generalized to adaptive control for handling exogenous disturbances in the follower vehicles. For each methodology, the closed-loop system stability properties are rigorously analyzed, the effect of the user-defined event-triggering thresholds and the controller design parameters on the overall system performance are characterized, and Zeno behavior is shown not to occur with the proposed algorithms --Abstract, page iv