253 research outputs found
Recent Research in Cooperative Control of Multivehicle Systems
This paper presents a survey of recent research in cooperative control of multivehicle systems, using a common mathematical framework to allow different methods to be described in a unified way. The survey has three primary parts: an overview of current applications of cooperative control, a summary of some of the key technical approaches that have been explored, and a description of some possible future directions for research. Specific technical areas that are discussed include formation control, cooperative tasking, spatiotemporal planning, and consensus
Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control
Decentralized multi-agent control has broad applications, ranging from
multi-robot cooperation to distributed sensor networks. In decentralized
multi-agent control, systems are complex with unknown or highly uncertain
dynamics, where traditional model-based control methods can hardly be applied.
Compared with model-based control in control theory, deep reinforcement
learning (DRL) is promising to learn the controller/policy from data without
the knowing system dynamics. However, to directly apply DRL to decentralized
multi-agent control is challenging, as interactions among agents make the
learning environment non-stationary. More importantly, the existing multi-agent
reinforcement learning (MARL) algorithms cannot ensure the closed-loop
stability of a multi-agent system from a control-theoretic perspective, so the
learned control polices are highly possible to generate abnormal or dangerous
behaviors in real applications. Hence, without stability guarantee, the
application of the existing MARL algorithms to real multi-agent systems is of
great concern, e.g., UAVs, robots, and power systems, etc. In this paper, we
aim to propose a new MARL algorithm for decentralized multi-agent control with
a stability guarantee. The new MARL algorithm, termed as a multi-agent
soft-actor critic (MASAC), is proposed under the well-known framework of
"centralized-training-with-decentralized-execution". The closed-loop stability
is guaranteed by the introduction of a stability constraint during the policy
improvement in our MASAC algorithm. The stability constraint is designed based
on Lyapunov's method in control theory. To demonstrate the effectiveness, we
present a multi-agent navigation example to show the efficiency of the proposed
MASAC algorithm.Comment: Accepted to The 2nd International Conference on Distributed
Artificial Intelligenc
An event-driven approach to control and optimization of multi-agent systems
This dissertation studies the application of several event-driven control schemes in multi-agent systems. First, a new cooperative receding horizon (CRH) controller is designed and applied to a class of maximum reward collection problems. Target rewards are time-variant with finite deadlines and the environment contains uncertainties. The new methodology adapts an event-driven approach by optimizing the control for a planning horizon and updating it for a shorter action horizon. The proposed CRH controller addresses several issues including potential instabilities and oscillations. It also improves the estimated reward-to-go which enhances the overall performance of the controller. The other major contribution is that the originally infinite-dimensional feasible control set is reduced to a finite set at each time step which improves the computational cost of the controller.
Second, a new event-driven methodology is studied for trajectory planning in multi-agent systems. A rigorous optimal control solution is employed using numerical solutions which turn out to be computationally infeasible in real time applications. The problem is then parameterized using several families of parametric trajectories. The solution to the parametric optimization relies on an unbiased estimate of the objective function's gradient obtained by the "Infinitesimal Perturbation Analysis" method. The premise of event-driven methods is that the events involved are observable so as to "excite" the underlying event-driven controller. However, it is not always obvious that these events actually take place under every feasible control in which case the controller may be useless. This issue of event excitation, which arises specially in multi-agent systems with a finite number of targets, is studied and addressed by introducing a novel performance measure which generates a potential field over the mission space. The effect of the new performance metric is demonstrated through simulation and analytical results
An event-driven approach to control and optimization of multi-agent systems
This dissertation studies the application of several event-driven control schemes in multi-agent systems. First, a new cooperative receding horizon (CRH) controller is designed and applied to a class of maximum reward collection problems. Target rewards are time-variant with finite deadlines and the environment contains uncertainties. The new methodology adapts an event-driven approach by optimizing the control for a planning horizon and updating it for a shorter action horizon. The proposed CRH controller addresses several issues including potential instabilities and oscillations. It also improves the estimated reward-to-go which enhances the overall performance of the controller. The other major contribution is that the originally infinite-dimensional feasible control set is reduced to a finite set at each time step which improves the computational cost of the controller.
Second, a new event-driven methodology is studied for trajectory planning in multi-agent systems. A rigorous optimal control solution is employed using numerical solutions which turn out to be computationally infeasible in real time applications. The problem is then parameterized using several families of parametric trajectories. The solution to the parametric optimization relies on an unbiased estimate of the objective function's gradient obtained by the "Infinitesimal Perturbation Analysis" method. The premise of event-driven methods is that the events involved are observable so as to "excite" the underlying event-driven controller. However, it is not always obvious that these events actually take place under every feasible control in which case the controller may be useless. This issue of event excitation, which arises specially in multi-agent systems with a finite number of targets, is studied and addressed by introducing a novel performance measure which generates a potential field over the mission space. The effect of the new performance metric is demonstrated through simulation and analytical results
An Overview of Recent Progress in the Study of Distributed Multi-agent Coordination
This article reviews some main results and progress in distributed
multi-agent coordination, focusing on papers published in major control systems
and robotics journals since 2006. Distributed coordination of multiple
vehicles, including unmanned aerial vehicles, unmanned ground vehicles and
unmanned underwater vehicles, has been a very active research subject studied
extensively by the systems and control community. The recent results in this
area are categorized into several directions, such as consensus, formation
control, optimization, task assignment, and estimation. After the review, a
short discussion section is included to summarize the existing research and to
propose several promising research directions along with some open problems
that are deemed important for further investigations
OPTIMAL LEADER-FOLLOWER FORMATION CONTROL USING DYNAMIC GAMES
Formation control is one of the salient features of multi-agent robotics. The main
goal of this field is to develop distributed control methods for interconnected multi-robot systems so that robots will move with respect to each other in order to keep a
formation throughout their joint mission. Numerous advantages and vast engineering
applications have drawn a great deal of attention to the research in this field.
Dynamic game theory is a powerful method to study dynamic interactions among
intelligent, rational, and self-interested agents. Differential game is among the most
important sub-classes of dynamic games, because many important problems in engineering
can be modeled as differential games.
The underlying goal of this research is to develop a reliable formation control
algorithm for multi-robot systems based on differential games. The main idea is to benefit from powerful machinery provided by dynamic games, and design an improved
formation control scheme with careful attention to practical control design requirements,
namely state feedback, and computation costs associated to implementation.
In this work, results from algebraic graph theory is used to develop a quasi-static optimal
control for heterogeneous leader{follower formation problem. The simulations
are provided to study capabilities as well as limitations associated to this approach.
Based on the obtained results, a finite horizon open-loop Nash differential game is
developed as adaptation of differential games methodology to formation control problems
in multi-robot systems. The practical control design requirements dictate state-feedback;
therefore, proposed controller is complimented by adding receding horizon
approach to its algorithm. It leads to a closed loop state-feedback formation control.
The simulation results are presented to show the effectiveness of proposed control
scheme
- …