Distributed Optimal Control and Application to Consensus of Multi-Agent Systems

This paper develops a novel approach to the consensus problem of multi-agent systems by minimizing a weighted state error with neighbor agents via linear quadratic (LQ) optimal control theory. Existing consensus control algorithms only utilize the current state of each agent, and the design of distributed controller depends on nonzero eigenvalues of the communication topology. The presented optimal consensus controller is obtained by solving Riccati equations and designing appropriate observers to account for agents' historical state information. It is shown that the corresponding cost function under the proposed controllers is asymptotically optimal. Simulation examples demonstrate the effectiveness of the proposed scheme, and a much faster convergence speed than the conventional consensus methods. Moreover, the new method avoids computing nonzero eigenvalues of the communication topology as in the traditional consensus methods

Supportive consensus

[EN] The paper is concerned with the consensus problem in a multi-agent system such that each agent has boundary constraints. Classical Olfati-Saber's consensus algorithm converges to the same value of the consensus variable, and all the agents reach the same value. These algorithms find an equality solution. However, what happens when this equality solution is out of the range of some of the agents? In this case, this solution is not adequate for the proposed problem. In this paper, we propose a new kind of algorithms called supportive consensus where some agents of the network can compensate for the lack of capacity of other agents to reach the average value, and so obtain an acceptable solution for the proposed problem. Supportive consensus finds an equity solution. In the rest of the paper, we define the supportive consensus, analyze and demonstrate the network's capacity to compensate out of boundaries agents, propose different supportive consensus algorithms, and finally, provide some simulations to show the performance of the proposed algorithms.The author(s) received specific funding for this work from the Valencian Research Institute for Artificial Intelligence (VRAIN) where the authors are currently working. This work is partially supported by the Spanish Government project RTI2018-095390-B-C31, GVA-CEICE project PROMETEO/2018/002, and TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Palomares Chust, A.; Rebollo Pedruelo, M.; Carrascosa Casamayor, C. (2020). Supportive consensus. PLoS ONE. 15(12):1-30. https://doi.org/10.1371/journal.pone.0243215S1301512Olfati-Saber, R., Fax, J. A., & Murray, R. M. (2007). Consensus and Cooperation in Networked Multi-Agent Systems. Proceedings of the IEEE, 95(1), 215-233. doi:10.1109/jproc.2006.887293Pérez, I. J., Cabrerizo, F. J., Alonso, S., Dong, Y. C., Chiclana, F., & Herrera-Viedma, E. (2018). On dynamic consensus processes in group decision making problems. Information Sciences, 459, 20-35. doi:10.1016/j.ins.2018.05.017Fischbacher, U., & Gächter, S. (2010). Social Preferences, Beliefs, and the Dynamics of Free Riding in Public Goods Experiments. American Economic Review, 100(1), 541-556. doi:10.1257/aer.100.1.541Du, S., Hu, L., & Song, M. (2016). Production optimization considering environmental performance and preference in the cap-and-trade system. Journal of Cleaner Production, 112, 1600-1607. doi:10.1016/j.jclepro.2014.08.086Alfonso, B., Botti, V., Garrido, A., & Giret, A. (2013). A MAS-based infrastructure for negotiation and its application to a water-right market. Information Systems Frontiers, 16(2), 183-199. doi:10.1007/s10796-013-9443-8Rebollo M, Carrascosa C, Palomares A. Consensus in Smart Grids for Decentralized Energy Management. In: Highlights of Practical Applications of Heterogeneous Multi-Agent Systems. The PAAMS Collection. Springer; 2014. p. 250–261.Zhao, T., & Ding, Z. (2018). Distributed Agent Consensus-Based Optimal Resource Management for Microgrids. IEEE Transactions on Sustainable Energy, 9(1), 443-452. doi:10.1109/tste.2017.2740833Qiu, Z., Liu, S., & Xie, L. (2018). Necessary and sufficient conditions for distributed constrained optimal consensus under bounded input. International Journal of Robust and Nonlinear Control, 28(6), 2619-2635. doi:10.1002/rnc.4040Wei Ren, & Beard, R. W. (2005). Consensus seeking in multiagent systems under dynamically changing interaction topologies. IEEE Transactions on Automatic Control, 50(5), 655-661. doi:10.1109/tac.2005.846556Ren, W., & Beard, R. W. (2008). Distributed Consensus in Multi-vehicle Cooperative Control. Communications and Control Engineering. doi:10.1007/978-1-84800-015-5Knorn F, Corless MJ, Shorten RN. A result on implicit consensus with application to emissions control. In: 2011 50th IEEE Conference on Decision and Control and European Control Conference; 2011. p. 1299–1304.Roy, S. (2015). Scaled consensus. Automatica, 51, 259-262. doi:10.1016/j.automatica.2014.10.073Mo, L., & Lin, P. (2018). Distributed consensus of second-order multiagent systems with nonconvex input constraints. International Journal of Robust and Nonlinear Control, 28(11), 3657-3664. doi:10.1002/rnc.4076Wang, Q., Gao, H., Alsaadi, F., & Hayat, T. (2014). An overview of consensus problems in constrained multi-agent coordination. Systems Science & Control Engineering, 2(1), 275-284. doi:10.1080/21642583.2014.897658Xi, J., Yang, J., Liu, H., & Zheng, T. (2018). Adaptive guaranteed-performance consensus design for high-order multiagent systems. Information Sciences, 467, 1-14. doi:10.1016/j.ins.2018.07.069Fontan A, Shi G, Hu X, Altafini C. Interval consensus: A novel class of constrained consensus problems for multiagent networks. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC); 2017. p. 4155–4160.Hou, W., Wu, Z., Fu, M., & Zhang, H. (2018). Constrained consensus of discrete-time multi-agent systems with time delay. International Journal of Systems Science, 49(5), 947-953. doi:10.1080/00207721.2018.1433899Elhage N, Beal J. Laplacian-based consensus on spatial computers. In: AAMAS; 2010. p. 907–914.Cavalcante R, Rogers A, Jennings N. Consensus acceleration in multiagent systems with the Chebyshev semi-iterative method. In: Proc. of AAMAS’11; 2011. p. 165–172.Hu, H., Yu, L., Zhang, W.-A., & Song, H. (2013). Group consensus in multi-agent systems with hybrid protocol. Journal of the Franklin Institute, 350(3), 575-597. doi:10.1016/j.jfranklin.2012.12.020Ji, Z., Lin, H., & Yu, H. (2012). Leaders in multi-agent controllability under consensus algorithm and tree topology. Systems & Control Letters, 61(9), 918-925. doi:10.1016/j.sysconle.2012.06.003Li, Y., & Tan, C. (2019). A survey of the consensus for multi-agent systems. Systems Science & Control Engineering, 7(1), 468-482. doi:10.1080/21642583.2019.1695689Salazar, N., Rodriguez-Aguilar, J. A., & Arcos, J. L. (2010). Robust coordination in large convention spaces. AI Communications, 23(4), 357-372. doi:10.3233/aic-2010-0479Pedroche F, Rebollo M, Carrascosa C, Palomares A. On the convergence of weighted-average consensus. CoRR. 2013;abs/1307.7562

Sampling-Based Optimization for Multi-Agent Model Predictive Control

We systematically review the Variational Optimization, Variational Inference and Stochastic Search perspectives on sampling-based dynamic optimization and discuss their connections to state-of-the-art optimizers and Stochastic Optimal Control (SOC) theory. A general convergence and sample complexity analysis on the three perspectives is provided through the unifying Stochastic Search perspective. We then extend these frameworks to their distributed versions for multi-agent control by combining them with consensus Alternating Direction Method of Multipliers (ADMM) to decouple the full problem into local neighborhood-level ones that can be solved in parallel. Model Predictive Control (MPC) algorithms are then developed based on these frameworks, leading to fully decentralized sampling-based dynamic optimizers. The capabilities of the proposed algorithms framework are demonstrated on multiple complex multi-agent tasks for vehicle and quadcopter systems in simulation. The results compare different distributed sampling-based optimizers and their centralized counterparts using unimodal Gaussian, mixture of Gaussians, and stein variational policies. The scalability of the proposed distributed algorithms is demonstrated on a 196-vehicle scenario where a direct application of centralized sampling-based methods is shown to be prohibitive

Tailoring Gradient Methods for Differentially-Private Distributed Optimization

Decentralized optimization is gaining increased traction due to its widespread applications in large-scale machine learning and multi-agent systems. The same mechanism that enables its success, i.e., information sharing among participating agents, however, also leads to the disclosure of individual agents' private information, which is unacceptable when sensitive data are involved. As differential privacy is becoming a de facto standard for privacy preservation, recently results have emerged integrating differential privacy with distributed optimization. Although such differential-privacy based privacy approaches for distributed optimization are efficient in both computation and communication, directly incorporating differential privacy design in existing distributed optimization approaches significantly compromises optimization accuracy. In this paper, we propose to redesign and tailor gradient methods for differentially-private distributed optimization, and propose two differential-privacy oriented gradient methods that can ensure both privacy and optimality. We prove that the proposed distributed algorithms can ensure almost sure convergence to an optimal solution under any persistent and variance-bounded differential-privacy noise, which, to the best of our knowledge, has not been reported before. The first algorithm is based on static-consensus based gradient methods and only shares one variable in each iteration. The second algorithm is based on dynamic-consensus (gradient-tracking) based distributed optimization methods and, hence, it is applicable to general directed interaction graph topologies. Numerical comparisons with existing counterparts confirm the effectiveness of the proposed approaches

Defending distributed systems against adversarial attacks: consensus, consensus-based learning, and statistical learning

A distributed system consists of networked components that interact with each other in order to achieve a common goal. Given the ubiquity of distributed systems and their vulnerability to adversarial attacks, it is crucial to design systems that are provably secured. In this dissertation, we propose and explore the problems of performing consensus, consensus-based learning, and statistical learning in the presence of malicious components. (1) Consensus: In this dissertation, we explore the influence of communication range on the computability of reaching iterative approximate consensus. Particularly, we characterize the tight topological condition on the networks for consensus to be achievable in the presence of Byzantine components. Our results bridge the gap of previous work. (2) Consensus-Based Learning: We propose, to the best of our knowledge, consensus-based Byzantine-tolerant learning problems: Consensus-Based Multi-Agent Optimization and Consensus-Based Distributed Hypothesis Testing. For the former, we characterize the performance degradation, and design efficient algorithms that can achieve the optimal fault-tolerance performance. For the latter, we propose, as far as we know, the first learning algorithm under which the good agents can collaboratively identify the underlying truth. (3) Statistical Learning: Finally, we explore distributed statistical learning, where the distributed system is captured by the server-client model. We develop a distributed machine learning algorithm that is able to (1) tolerate Byzantine failures, (2) accurately learn a highly complex model with low local data volume, and (3) converge exponentially fast using logarithmic communication rounds

COOPERATIVE AND CONSENSUS-BASED CONTROL FOR A TEAM OF MULTI-AGENT SYSTEMS

Cooperative control has attracted a noticeable interest in control systems community due to its numerous applications in areas such as formation flying of unmanned aerial vehicles, cooperative attitude control of spacecraft, rendezvous of mobile robots, unmanned underwater vehicles, traffic control, data network congestion control and routing. Generally, in any cooperative control of multi-agent systems one can find a set of locally sensed information, a communication network with limited bandwidth, a decision making algorithm, and a distributed computational capability. The ultimate goal of cooperative systems is to achieve consensus or synchronization throughout the team members while meeting all communication and computational constraints. The consensus problem involves convergence of outputs or states of all agents to a common value and it is more challenging when the agents are subjected to disturbances, measurement noise, model uncertainties or they are faulty. This dissertation deals with the above mentioned challenges and has developed methods to design distributed cooperative control and fault recovery strategies in multi-agent systems. Towards this end, we first proposed a transformation for Linear Time Invariant (LTI) multi-agent systems that facilitates a systematic control design procedure and make it possible to use powerful Lyapunov stability analysis tool to guarantee its consensus achievement. Moreover, Lyapunov stability analysis techniques for switched systems are investigated and a novel method is introduced which is well suited for designing consensus algorithms for switching topology multi-agent systems. This method also makes it possible to deal with disturbances with limited root mean square (RMS) intensities. In order to decrease controller design complexity, a iii method is presented which uses algebraic connectivity of the communication network to decouple augmented dynamics of the team into lower dimensional parts, which allows one to design the consensus algorithm based on the solution to an algebraic Riccati equation with the same order as that of agent. Although our proposed decoupling method is a powerful approach to reduce the complexity of the controller design, it is possible to apply classical pole placement methods to the transformed dynamics of the team to develop and obtain controller gains. The effects of actuator faults in consensus achievement of multi-agent systems is investigated. We proposed a framework to quantitatively study actuator loss-of-effectiveness effects in multi-agent systems. A fault index is defined based on information on fault severities of agents and communication network topology, and sufficient conditions for consensus achievement of the team are derived. It is shown that the stability of the cooperative controller is linked to the fault index. An optimization problem is formulated to minimize the team fault index that leads to improvements in the performance of the team. A numerical optimization algorithm is used to obtain the solutions to the optimal problem and based on the solutions a fault recovery strategy is proposed for both actuator saturation and loss-of-effectiveness fault types. Finally, to make our proposed methodology more suitable for real life scenarios, the consensus achievement of a multi-agent team in presence of measurement noise and model uncertainties is investigated. Towards this end, first a team of LTI agents with measurement noise is considered and an observer based consensus algorithm is proposed and shown that the team can achieve H∞ output consensus in presence of both bounded RMS disturbance input and measurement noise. In the next step a multi-agent team with both linear and Lipschitz nonlinearity uncertainties is studied and a cooperative control algorithm is developed. An observer based approach is also developed to tackle consensus achievement problem in presence of both measurement noise and model uncertainties