Overcoming local optima in control and optimization of cooperative multi-agent systems

A cooperative multi-agent system is a collection of interacting agents deployed in a mission space where each agent is allowed to control its local state so that the fleet of agents collectively optimizes a common global objective. While optimization problems associated with multi-agent systems intend to determine the fixed set of globally optimal agent states, control problems aim to obtain the set of globally optimal agent controls. Associated non-convexities in these problems result in multiple local optima. This dissertation explores systematic techniques that can be deployed to either escape or avoid poor local optima while in search of provably better (still local) optima. First, for multi-agent optimization problems with iterative gradient-based solutions, a distributed approach to escape local optima is proposed based on the concept of boosting functions. These functions temporarily transform gradient components at a local optimum into a set of boosted non-zero gradient components in a systematic manner so that it is more effective compared to the methods where gradient components are randomly perturbed. A novel variable step size adjustment scheme is also proposed to establish the convergence of this distributed boosting process. Developed boosting concepts are successfully applied to the class of coverage problems. Second, as a means of avoiding convergence to poor local optima in multi-agent optimization, the use of greedy algorithms in generating effective initial conditions is explored. Such greedy methods are computationally cheap and can often exploit submodularity properties of the problem to provide performance bound guarantees to the obtained solutions. For the class of submodular maximization problems, two new performance bounds are proposed and their effectiveness is illustrated using the class of coverage problems. Third, a class of multi-agent control problems termed Persistent Monitoring on Networks (PMN) is considered where a team of agents is traversing a set of nodes (targets) interconnected according to a network topology aiming to minimize a measure of overall node state. For this class of problems, a gradient-based parametric control solution developed in a prior work relies heavily on the initial selection of its `parameters' which often leads to poor local optima. To overcome this initialization challenge, the PMN system's asymptotic behavior is analyzed, and an off-line greedy algorithm is proposed to systematically generate an effective set of initial parameters. Finally, for the same class of PMN problems, a computationally efficient distributed on-line Event-Driven Receding Horizon Control (RHC) solution is proposed as an alternative. This RHC solution is parameter-free as it automatically optimizes its planning horizon length and gradient-free as it uses explicitly derived solutions for each RHC problem invoked at each agent upon each event of interest. Hence, unlike the gradient-based parametric control solutions, the proposed RHC solution does not force the agents to converge to one particular behavior that is likely to be a poor local optimum. Instead, it keeps the agents actively searching for the optimum behavior. In each of these four parts of the thesis, an interactive simulation platform is developed (and made available online) to generate extensive numerical examples that highlight the respective contributions made compared to the state of the art

A Generalized Distributed Analysis and Control Synthesis Approach for Networked Systems with Arbitrary Interconnections

We consider the problem of distributed analysis and control synthesis to verify and ensure properties like stability and dissipativity of a large-scale networked system comprised of linear subsystems interconnected in an arbitrary topology. In particular, we design systematic networked system analysis and control synthesis processes that can be executed in a distributed manner at the subsystem level with minimal information sharing among the subsystems. Compared to a recent work on the same topic, we consider a substantially more generalized problem setup and develop distributed processes to verify and ensure a broader range of networked system properties. We also show that optimizing subsystems' indexing scheme used in such distributed processes can substantially reduce the required information-sharing sessions between subsystems. Moreover, the proposed networked system analysis and control synthesis processes are compositional and thus allow them to conveniently and efficiently handle situations where new subsystems are being added to an existing network. We also provide significant insights into our approach so that it can be quickly adopted to verify and ensure properties beyond the stability and dissipativity of networked systems. Finally, we provide several simulation results to demonstrate the proposed distributed analysis and control synthesis processes.Comment: To be presented in the 30th Mediterranean Conference on Control and Automation, Athens, Greece 202

Distributed non-convex optimization of multi-agent systems using boosting functions to escape local optima

We address the problem of multiple local optima arising in cooperative multi-agent optimization problems with non-convex objective functions. We propose a systematic approach to escape these local optima using boosting functions. These functions temporarily transform a gradient at a local optimum into a "boosted" non-zero gradient. Extending a prior centralized optimization approach, we develop a distributed framework for the use of boosted gradients and show that convergence of this distributed process can be attained by employing an optimal variable step size scheme for gradient-based algorithms. Numerical examples are included to show how the performance of a class of multi-agent optimization systems can be improved.Accepted manuscrip

Optimal composition of heterogeneous multi-agent teams for coverage problems with performance bound guarantees

First author draf

Non-intrusive load monitoring under residential solar power influx

This paper proposes a novel Non-Intrusive Load Monitoring (NILM) method for a consumer premises with a residentially installed solar plant. This method simultaneously identifies the amount of solar power influx as well as the turned ON appliances, their operating modes, and power consumption levels. Further, it works effectively with a single active power measurement taken at the total power entry point with a sampling rate of 1 Hz. First, a unique set of appliance and solar signatures were constructed using a high-resolution implementation of Karhunen Loéve expansion (KLE). Then, different operating modes of multi-state appliances were automatically classified utilizing a spectral clustering based method. Finally, using the total power demand profile, through a subspace component power level matching algorithm, the turned ON appliances along with their operating modes and power levels as well as the solar influx amount were found at each time point. The proposed NILM method was first successfully validated on six synthetically generated houses (with solar units) using real household data taken from the Reference Energy Disaggregation Dataset (REDD) - USA. Then, in order to demonstrate the scalability of the proposed NILM method, it was employed on a set of 400 individual households. From that, reliable estimations were obtained for the total residential solar generation and for the total load that can be shed to provide reserve services. Finally, through a developed prediction technique, NILM results observed from 400 households during four days in the recent past were utilized to predict the next day’s total load that can be shed

Event-Driven Receding Horizon Control For Distributed Persistent Monitoring on Graphs

We consider the optimal multi-agent persistent monitoring problem defined on a set of nodes (targets) inter-connected through a fixed graph topology. The objective is to minimize a measure of mean overall node state uncertainty evaluated over a finite time interval by controlling the motion of a team of agents. Prior work has addressed this problem through on-line parametric controllers and gradient-based methods, often leading to low-performing local optima or through off-line computationally intensive centralized approaches. This paper proposes a computationally efficient event-driven receding horizon control approach providing a distributed on-line gradient-free solution to the persistent monitoring problem. A novel element in the controller, which also makes it parameter-free, is that it self-optimizes the planning horizon over which control actions are sequentially taken in event-driven fashion. Numerical results show significant improvements compared to state of the art distributed on-line parametric control solutions.Accepted manuscrip

Smooth Robustness Measures for Symbolic Control Via Signal Temporal Logic

Symbolic control problems aim to synthesize control policies for dynamical systems under complex temporal specifications. For such problems, Signal Temporal Logic (STL) is increasingly used as the formal specification language due to its rich expressiveness. Moreover, the degree of satisfaction of STL specifications can be evaluated using ``STL robust semantics'' as a scalar robustness measure. This capability of STL enables transforming a symbolic control problem into an optimization problem that optimizes the corresponding robustness measure. However, since these robustness measures are non-smooth and non-convex, exact solutions can only be computed using computationally inefficient mixed-integer programming techniques that do not scale well. Therefore, recent literature has focused on using smooth approximations of these robustness measures to apply scalable and computationally efficient gradient-based methods to find local optima solutions. In this paper, we first generalize two recently established smooth robustness measures (SRMs) and two new ones and discuss their strengths and weaknesses. Next, we propose ``STL error semantics'' to characterize the approximation errors associated with different SRMs under different parameter configurations. This allows one to sensibly select an SRM (to optimize) along with its parameter values. We then propose ``STL gradient semantics'' to derive explicit gradients of SRMs leading to improve computational efficiency as well as accuracy compared to when using numerically estimated gradients. Finally, these contributions are highlighted using extensive simulation results.Comment: To be submitted to ACC 2024 and TA