On the genericity properties in networked estimation: Topology design and sensor placement

In this paper, we consider networked estimation of linear, discrete-time dynamical systems monitored by a network of agents. In order to minimize the power requirement at the (possibly, battery-operated) agents, we require that the agents can exchange information with their neighbors only \emph{once per dynamical system time-step}; in contrast to consensus-based estimation where the agents exchange information until they reach a consensus. It can be verified that with this restriction on information exchange, measurement fusion alone results in an unbounded estimation error at every such agent that does not have an observable set of measurements in its neighborhood. To over come this challenge, state-estimate fusion has been proposed to recover the system observability. However, we show that adding state-estimate fusion may not recover observability when the system matrix is structured-rank ($S$-rank) deficient. In this context, we characterize the state-estimate fusion and measurement fusion under both full $S$-rank and $S$-rank deficient system matrices.Comment: submitted for IEEE journal publicatio

Consensus tracking of nonlinear agents using distributed nonlinear dynamic inversion with switching leader-follower connection

In this paper, a consensus tracking protocol for nonlinear agents is presented, which is based on the Nonlinear Dynamic Inversion (NDI) technique. Implementation of such a technique is new in the context of the consensus tracking problem. The tracking capability of nonlinear dynamic inversion (NDI) is exploited for a leader-follower multi-agent scenario. We have provided all the mathematical details to establish its theoretical foundation. Additionally, a convergence study is provided to show the efficiency of the proposed controller. The performance of the proposed controller is evaluated in the presence of both (a) random switching topology among the agents and (b) random switching of leader–follower connections, which is realistic and not reported in the literature. The follower agents track various trajectories generated by a dynamic leader, which describes the tracking capability of the proposed controller. The results obtained from the simulation study show how efficiently this controller can handle the switching topology and switching leader-follower connections.Engineering and Physical Sciences Research Council (EPSRC): EP/R009953/

Consensus of Multi-agent Reinforcement Learning Systems: The Effect of Immediate Rewards

This paper studies the consensus problem of a leaderless, homogeneous, multi-agent reinforcement learning (MARL) system using actor-critic algorithms with and without malicious agents. The goal of each agent is to reach the consensus position with the maximum cumulative reward. Although the reward function converges in both scenarios, in the absence of the malicious agent, the cumulative reward is higher than with the malicious agent present. We consider here various immediate reward functions. First, we study the immediate reward function based on Manhattan distance. In addition to proposing three different immediate reward functions based on Euclidean, $n$-norm, and Chebyshev distances, we have rigorously shown which method has a better performance based on a cumulative reward for each agent and the entire team of agents. Finally, we present a combination of various immediate reward functions that yields a higher cumulative reward for each agent and the team of agents. By increasing the agents’ cumulative reward using the combined immediate reward function, we have demonstrated that the cumulative team reward in the presence of a malicious agent is comparable with the cumulative team reward in the absence of the malicious agent. The claims have been proven theoretically, and the simulation confirms theoretical findings

Coordination of multi-agent systems: stability via nonlinear Perron-Frobenius theory and consensus for desynchronization and dynamic estimation.

This thesis addresses a variety of problems that arise in the study of complex networks composed by multiple interacting agents, usually called multi-agent systems (MASs). Each agent is modeled as a dynamical system whose dynamics is fully described by a state-space representation. In the first part the focus is on the application to MASs of recent results that deal with the extensions of Perron-Frobenius theory to nonlinear maps. In the shift from the linear to the nonlinear framework, Perron-Frobenius theory considers maps being order-preserving instead of matrices being nonnegative. The main contribution is threefold. First of all, a convergence analysis of the iterative behavior of two novel classes of order-preserving nonlinear maps is carried out, thus establishing sufficient conditions which guarantee convergence toward a fixed point of the map: nonnegative row-stochastic matrices turns out to be a special case. Secondly, these results are applied to MASs, both in discrete and continuous-time: local properties of the agents' dynamics have been identified so that the global interconnected system falls into one of the above mentioned classes, thus guaranteeing its global stability. Lastly, a sufficient condition on the connectivity of the communication network is provided to restrict the set of equilibrium points of the system to the consensus points, thus ensuring the agents to achieve consensus. These results do not rely on standard tools (e.g., Lyapunov theory) and thus they constitute a novel approach to the analysis and control of multi-agent dynamical systems. In the second part the focus is on the design of dynamic estimation algorithms in large networks which enable to solve specific problems. The first problem consists in breaking synchronization in networks of diffusively coupled harmonic oscillators. The design of a local state feedback that achieves desynchronization in connected networks with arbitrary undirected interactions is provided. The proposed control law is obtained via a novel protocol for the distributed estimation of the Fiedler vector of the Laplacian matrix. The second problem consists in the estimation of the number of active agents in networks wherein agents are allowed to join or leave. The adopted strategy consists in the distributed and dynamic estimation of the maximum among numbers locally generated by the active agents and the subsequent inference of the number of the agents that took part in the experiment. Two protocols are proposed and characterized to solve the consensus problem on the time-varying max value. The third problem consists in the average state estimation of a large network of agents where only a few agents' states are accessible to a centralized observer. The proposed strategy projects the dynamics of the original system into a lower dimensional state space, which is useful when dealing with large-scale systems. Necessary and sufficient conditions for the existence of a linear and a sliding mode observers are derived, along with a characterization of their design and convergence properties