Cooperative AI via Decentralized Commitment Devices

Credible commitment devices have been a popular approach for robust multi-agent coordination. However, existing commitment mechanisms face limitations like privacy, integrity, and susceptibility to mediator or user strategic behavior. It is unclear if the cooperative AI techniques we study are robust to real-world incentives and attack vectors. However, decentralized commitment devices that utilize cryptography have been deployed in the wild, and numerous studies have shown their ability to coordinate algorithmic agents facing adversarial opponents with significant economic incentives, currently in the order of several million to billions of dollars. In this paper, we use examples in the decentralization and, in particular, Maximal Extractable Value (MEV) (arXiv:1904.05234) literature to illustrate the potential security issues in cooperative AI. We call for expanded research into decentralized commitments to advance cooperative AI capabilities for secure coordination in open environments and empirical testing frameworks to evaluate multi-agent coordination ability given real-world commitment constraints.Comment: NeurIPS 2023- Multi-Agent Security Worksho

Courtesy as a Means to Coordinate

We investigate the problem of multi-agent coordination under rationality constraints. Specifically, role allocation, task assignment, resource allocation, etc. Inspired by human behavior, we propose a framework (CA^3NONY) that enables fast convergence to efficient and fair allocations based on a simple convention of courtesy. We prove that following such convention induces a strategy which constitutes an $\epsilon$-subgame-perfect equilibrium of the repeated allocation game with discounting. Simulation results highlight the effectiveness of CA^3NONY as compared to state-of-the-art bandit algorithms, since it achieves more than two orders of magnitude faster convergence, higher efficiency, fairness, and average payoff.Comment: Accepted at AAMAS 2019 (International Conference on Autonomous Agents and Multiagent Systems

Multi-Armed Bandits for Spectrum Allocation in Multi-Agent Channel Bonding WLANs

While dynamic channel bonding (DCB) is proven to boost the capacity of wireless local area networks (WLANs) by adapting the bandwidth on a per-frame basis, its performance is tied to the primary and secondary channel selection. Unfortunately, in uncoordinated high-density deployments where multiple basic service sets (BSSs) may potentially overlap, hand-crafted spectrum management techniques perform poorly given the complex hidden/exposed nodes interactions. To cope with such challenging Wi-Fi environments, in this paper, we first identify machine learning (ML) approaches applicable to the problem at hand and justify why model-free RL suits it the most. We then design a complete RL framework and call into question whether the use of complex RL algorithms helps the quest for rapid learning in realistic scenarios. Through extensive simulations, we derive that stateless RL in the form of lightweight multi-armed-bandits (MABs) is an efficient solution for rapid adaptation avoiding the definition of broad and/or meaningless states. In contrast to most current trends, we envision lightweight MABs as an appropriate alternative to the cumbersome and slowly convergent methods such as Q-learning, and especially, deep reinforcement learning

Energy management in plug-in hybrid electric vehicles: recent progress and a connected vehicles perspective

Plug-in hybrid electric vehicles (PHEVs) offer an immediate solution for emissions reduction and fuel displacement within the current infrastructure. Targeting PHEV powertrain optimization, a plethora of energy management strategies (EMSs) have been proposed. Although these algorithms present various levels of complexity and accuracy, they find a limitation in terms of availability of future trip information, which generally prevents exploitation of the full PHEV potential in real-life cycles. This paper presents a comprehensive analysis of EMS evolution toward blended mode (BM) and optimal control, providing a thorough survey of the latest progress in optimization-based algorithms. This is performed in the context of connected vehicles and highlights certain contributions that intelligent transportation systems (ITSs), traffic information, and cloud computing can provide to enhance PHEV energy management. The study is culminated with an analysis of future trends in terms of optimization algorithm development, optimization criteria, PHEV integration in the smart grid, and vehicles as part of the fleet

Multi-type Fair Resource Allocation for Distributed Multi-Robot Systems

Fair resource allocation is essential to ensure that all resource requesters acquire adequate resources and accomplish tasks. We propose solutions to the fairness problem in multi-type resource allocation for multi-robot systems that have multiple resource requesters. We apply the dominant resource fairness (DRF) principle in our solutions to two different systems: single-tasking robots with multi-robot tasks (STR-MRT) and multi-tasking robots with single-robot tasks (MTR-SRT). In STR-MRT, each robot can perform only one task at a time, tasks are divisible, and accomplishing each task requires one or more robots. In MTR-SRT, each robot can perform multiple tasks at a time, tasks are not divisible, and accomplishing each task requires only one robot. We present centralized solutions to the fairness problem in STR-MRT. Meanwhile, we model the decentralized resource allocation in STR-MRT as a coordination game between the robots. Each robot subgroup is formed by robots that strategically select the same resource requester. For a requester associated with a specific subgroup, a consensus-based team formation algorithm further chooses the minimal set of robots to accomplish the task. We leverage the Deep Q-learning Network (DQN) to support requester selection. The results suggest that the DQN outperforms the commonly used Q-learning. Finally, we propose two decentralized solutions to promote fair resource allocation in MTR-SRT, as a centralized solution already exists. We first propose a task-forwarding solution in which the robots need to negotiate the placement of each task. In our second solution, each robot first selects resource requesters and then independently allocates resources to tasks that arrive from the selected requesters. The resource-requester selection phase of the latter solution models a coordination game that is solved by reinforcement learning. The experimental results suggest that both approaches outperform their baselines

Optimal control of a motor-integrated hybrid powertrain for a two-wheeled vehicle suitable for personal transportation

The present research aims to propose an optimized configuration of the motor integrated power-train with an optimal controller suitable for small power-train based two wheeler automobile which can increase the system level efficiency without affecting drivability. This work will be the foundation for realizing the system in a production ready vehicle for the two wheeler OEM TVS Motor Company in India. A detailed power-train model is developed (from first principles) for the scooter vehicle, which is powered by a 110 cc spark ignition (SI) engine and coupled with two types of transmission, a continuous variable transmission (CVT) and a 4-speed manual transmission (MT). Both models are capable of simulating torque and NOx emission output of the SI engine and dynamic response of the full power-train. The torque production and emission outputs of the model are compared with experimental results available from TVS Motor Company. The CVT gear ratio model is developed using an indirect method and an analytical model. Both types of powertrain models are applied to perform a simulated study of fuel consumption, NOx emission and drivability study for a particular vehicle platform. In the next stage of work, the mathematical model for a brush-less direct current machine (BLDC) with the drive system and Li-Ion battery are developed. The models are verified and calibrated with the experimental results from TVS Motor Company. The BLDC machine is integrated with both the CVT and MT powertrain models in parallel hybrid configurations and a drive cycle simulation is conducted for different static assist levels by the electrical machines. The initial test confirms the need of optimal sizing of the powertrain components as well as an optimal control system. The detailed model of the powertrain is converted to a control-oriented model which is suitable for optimal control. This is followed by multi-objective optimization of different components of the motor-integrated powertrain using a single function as well as Pareto-Optimal methods. The objective function for the multi-objective optimization is proposed to reduce the fuel consumption with battery charge sustainability with least impact on the increase of financial cost and weight of the vehicle. The optimization is conducted by a nested methodology that involves Particle Swarm Optimization and a Non-dominated sorting genetic algorithm where, concurrently, a global optimal control is developed corresponding to the multi-objective design. The global optimal controller is designed using dynamic programming. The research is concluded with an optimal controller developed using the hp-collocation method. The objective function of the dynamic programming method and hp-collocation method is proposed to reduce fuel consumption with battery charge sustainability.Open Acces

Active strategies for coordination of solitary robots

Thesis (PhD)--Stellenbosch University, 2020.ENGLISH ABSTRACT: This thesis considers the problem of search of an unknown environment by multiple solitary robots: self-interested robots without prior knowledge about each other, and with restricted perception and communication capacity. When solitary robots accidentally interact with each other, they can leverage each other’s information to work more effectively. In this thesis, we consider three problems related to the treatment of solitary robots: coordination, construction of a view of the network formed when robots interact, and classifier fusion. Coordination is the key focus for search and rescue. The other two problems are related areas inspired by the problems we encountered while developing our coordination method. We propose a coordination strategy based on cellular decomposition of the search environment, which provides sustainable performance when a known available search time (bound) is insufficient to cover the entire search environment. A sustainable performance is achieved when robots that know about each other explore non-overlapping regions. For network construction, we propose modifications to a scalable decentralised method for constructing a model of network topology which reduces the number of messages exchanged between interacting nodes. The method has wider potential application than mobile robotics. For classifier fusion, we propose an iterative method where outputs of classifiers are combined without using any further information about the behaviour of the individual classifiers. Our approaches for each of these problems are compared to state-of-the-art methods.AFRIKAANSE OPSOMMING: Hierdie tesis beskou die probleem van soektog in ’n onbekende omgewing deur ’n aantal alleenstaande robotte: selfbelangstellende robotte sonder voorafgaande kennis van mekaar, en met beperkte persepsie- en kommunikasievermoëns. Wanneer alleenstaande robotte toevallig mekaar raakloop, kan hulle met mekaar inligting uitruil om meer effektief te werk. Hierdie tesis beskou drie probleme wat verband hou met die hantering van alleenstaande robotte: konstruksie van ’n blik van die netwerk gevorm deur interaksie tussen robotte, koördinasie en klassifiseerdersamesmelting. Koördinasie is die hoof fokuspunt vir soek en redding. Die ander twee probleme is uit verwante areas, gemotiveer deur uitdagings wat ons ervaar het tydens die ontwikkeling van ons koördineringsmetode. Ons stel ’n skaleerbare desentraliseerde metode voor om ’n model van netwerktopologie te bou wat minder boodskappe tussen wisselwerkende nodusse hoet te verruil. Die metode het wyer potensiële toepassings as mobiele robotika. Vir koördinasie, stel ons ’n strategie voor gebaseer op sellulêre ontbinding van die soekomgewing, wat volhoubare prestasie toon wanneer ’n bekende soektyd onvoldoende is om die hele soekomgewing te dek. Vir klassifiseerdersamesmelting, stel ons ’n iteratiewe metode voor, waar klassifiseerders se voorspellings gekombineer word sonder om enige verdere inligting oor die gedrag van die individuele klassifiseerders te gebruik. Ons benaderings vir elkeen van hierdie probleme word vergelyk met stand-van-die-kuns metodes.The financial assistance of the African Institute for Mathematical Sciences (AIMS) and CSIR-SU Centre for Artificial Intelligence Research Group (CSIR-SU CAIR) towards this research is hereby acknowledged. Opinions expressed and conclusions arrived at, are those of the author and are not necessarily to be attributed to the AIMS and CSIR-SU CAIR.Doctora

Decentralized Anti-coordination Through Multi-agent Learning

To achieve an optimal outcome in many situations, agents need to choose distinct actions from one another. This is the case notably in many resource allocation problems, where a single resource can only be used by one agent at a time. How shall a designer of a multi-agent system program its identical agents to behave each in a different way? From a game theoretic perspective, such situations lead to undesirable Nash equilibria. For example consider a resource allocation game in that two players compete for an exclusive access to a single resource. It has three Nash equilibria. The two pure-strategy NE are efficient, but not fair. The one mixed-strategy NE is fair, but not efficient. Aumann’s notion of correlated equilibrium fixes this problem: It assumes a correlation device that suggests each agent an action to take. However, such a “smart ” coordination device might not be available. We propose using a randomly chosen, “stupid ” integer coordination signal. “Smart ” agents learn which action they should use for each value of the coordination signal. We present a multi-agent learning algorithm that converges in polynomial number of steps to a correlated equilibrium of a channel allocation game, a variant of the resource allocation game. We show that the agents learn to play for each coordination signal value a randomly chosen pure-strategy Nash equilibrium of the game. Therefore, the outcome is an efficient correlated equilibrium. This CE becomes more fair as the number of the available coordination signal values increases. 1