Heuristic methods for coalition structure generation

The Coalition Structure Generation (CSG) problem requires finding an optimal partition of a set of n agents. An optimal partition means one that maximizes global welfare. Computing an optimal coalition structure is computationally hard especially when there are externalities, i.e., when the worth of a coalition is dependent on the organisation of agents outside the coalition. A number of algorithms were previously proposed to solve the CSG problem but most of these methods were designed for systems without externalities. Very little attention has been paid to finding optimal coalition structures in the presence of externalities, although externalities are a key feature of many real world multiagent systems. Moreover, the existing methods, being non-heuristic, have exponential time complexity which means that they are infeasible for any but systems comprised of a small number of agents. The aim of this research is to develop effective heuristic methods for finding optimal coalition structures in systems with externalities, where time taken to find a solution is more important than the quality of the solution. To this end, four different heuristics methods namely tabu search, simulated annealing, ant colony search and particle swarm optimisation are explored. In particular, neighbourhood operators were devised for the effective exploration of the search space and a compact representation method was formulated for storing details about the multiagent system. Using these, the heuristic methods were devised and their performance was evaluated extensively for a wide range of input data

Dynamic constrained coalition formation among electric vehicles

Background: The use of electric vehicles (EVs) and vehicle-to-grid (V2G) technologies have been advocated as an efficient way to reduce the intermittency of renewable energy sources in smart grids. However, operating on V2G sessions in a cost-effective way is not a trivial task for EVs. The formation of coalitions among EVs has been proposed to tackle this problem. Methods: In this paper we introduce Dynamic Constrained Coalition Formation (DCCF), which is a distributed heuristic-based method for constrained coalition structure generation (CSG) in dynamic environments. In our approach, coalitions are formed observing constraints imposed by the grid. To this end, EV agents negotiate the formation of feasible coalitions among themselves. Results: Based on experiments, we show that DCCF is efficient to provide good solutions in a fast way. DCCF provides solutions whose quality approaches 98% of the optimum. In dynamically changing scenarios, DCCF also shows good results, keeping the agents payoff stable along time. Conclusions: Essentially, DCCF’s main advantage over traditional CSG algorithms is that its computational effort is very lower. On the other hand, unlike traditional algorithms, DCCF is suitable only for constraint-based problems

Combining Optimization and Machine Learning for the Formation of Collectives

This thesis considers the problem of forming collectives of agents for real-world applications aligned with Sustainable Development Goals (e.g., shared mobility and cooperative learning). Such problems require fast approaches that can produce solutions of high quality for hundreds of agents. With this goal in mind, existing solutions for the formation of collectives focus on enhancing the optimization approach by exploiting the characteristics of a domain. However, the resulting approaches rely on specific domain knowledge and are not transferable to other collective formation problems. Therefore, approaches that can be applied to various problems need to be studied in order to obtain general approaches that do not require prior knowledge of the domain. Along these lines, this thesis proposes a general approach for the formation of collectives based on a novel combination of machine learning and an \emph{Integer Linear Program}. More precisely, a machine learning component is trained to generate a set of promising collectives that are likely to be part of a solution. Then, such collectives and their corresponding utility values are introduced into an \emph{Integer Linear Program} which finds a solution to the collective formation problem. In that way, the machine learning component learns the structure shared by ``good'' collectives in a particular domain, making the whole approach valid for various applications. In addition, the empirical analysis conducted on two real-world domains (i.e., ridesharing and team formation) shows that the proposed approach provides solutions of comparable quality to state-of-the-art approaches specific to each domain. Finally, this thesis also shows that the proposed approach can be extended to problems that combine the formation of collectives with other optimization objectives. Thus, this thesis proposes an extension of the collective formation approach for assigning pickup and delivery locations to robots in a warehouse environment. The experimental evaluation shows that, although it is possible to use the collective formation approach for that purpose, several improvements are required to compete with state-of-the-art approaches. Overall, this thesis aims to demonstrate that machine learning can be successfully intertwined with classical optimization approaches for the formation of collectives by learning the structure of a domain, reducing the need for ad-hoc algorithms devised for a specific application

Human-agent collectives

We live in a world where a host of computer systems, distributed throughout our physical and information environments, are increasingly implicated in our everyday actions. Computer technologies impact all aspects of our lives and our relationship with the digital has fundamentally altered as computers have moved out of the workplace and away from the desktop. Networked computers, tablets, phones and personal devices are now commonplace, as are an increasingly diverse set of digital devices built into the world around us. Data and information is generated at unprecedented speeds and volumes from an increasingly diverse range of sources. It is then combined in unforeseen ways, limited only by human imagination. People’s activities and collaborations are becoming ever more dependent upon and intertwined with this ubiquitous information substrate. As these trends continue apace, it is becoming apparent that many endeavours involve the symbiotic interleaving of humans and computers. Moreover, the emergence of these close-knit partnerships is inducing profound change. Rather than issuing instructions to passive machines that wait until they are asked before doing anything, we will work in tandem with highly inter-connected computational components that act autonomously and intelligently (aka agents). As a consequence, greater attention needs to be given to the balance of control between people and machines. In many situations, humans will be in charge and agents will predominantly act in a supporting role. In other cases, however, the agents will be in control and humans will play the supporting role. We term this emerging class of systems human-agent collectives (HACs) to reflect the close partnership and the flexible social interactions between the humans and the computers. As well as exhibiting increased autonomy, such systems will be inherently open and social. This means the participants will need to continually and flexibly establish and manage a range of social relationships. Thus, depending on the task at hand, different constellations of people, resources, and information will need to come together, operate in a coordinated fashion, and then disband. The openness and presence of many distinct stakeholders means participation will be motivated by a broad range of incentives rather than diktat. This article outlines the key research challenges involved in developing a comprehensive understanding of HACs. To illuminate this agenda, a nascent application in the domain of disaster response is presented

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Algorithms for Modular Self-reconfigurable Robots: Decision Making, Planning, and Learning

Modular self-reconfigurable robots (MSRs) are composed of multiple robotic modules which can change their connections with each other to take different shapes, commonly known as configurations. Forming different configurations helps the MSR to accomplish different types of tasks in different environments. In this dissertation, we study three different problems in MSRs: partitioning of modules, configuration formation planning and locomotion learning, and we propose algorithmic solutions to solve these problems. Partitioning of modules is a decision-making problem for MSRs where each module decides which partition or team of modules it should be in. To find the best set of partitions is a NP-complete problem. We propose game theory based both centralized and distributed solutions to solve this problem. Once the modules know which set of modules they should team-up with, they self-aggregate to form a specific shaped configuration, known as the configuration formation planning problem. Modules can be either singletons or connected in smaller configurations from which they need to form the target configuration. The configuration formation problem is difficult as multiple modules may select the same location in the target configuration to move to which might result in occlusion and consequently failure of the configuration formation process. On the other hand, if the modules are already in connected configurations in the beginning, then it would be beneficial to preserve those initial configurations for placing them into the target configuration as disconnections and re-connections are costly operations. We propose solutions based on an auction-like algorithm and (sub) graph-isomorphism technique to solve the configuration formation problem. Once the configuration is built, the MSR needs to move towards its goal location as a whole configuration for completing its task. If the configuration’s shape and size is not known a priori, then planning its locomotion is a difficult task as it needs to learn the locomotion pattern in dynamic time – the problem is known as adaptive locomotion learning. We have proposed reinforcement learning based fault-tolerant solutions for locomotion learning by MSRs

SB-CoRLA: Schema-Based Constructivist Robot Learning Architecture

This dissertation explores schema-based robot learning. I developed SB-CoRLA (Schema- Based, Constructivist Robot Learning Architecture) to address the issue of constructivist robot learning in a schema-based robot system. The SB-CoRLA architecture extends the previously developed ASyMTRe (Automated Synthesis of Multi-team member Task solutions through software Reconfiguration) architecture to enable constructivist learning for multi-robot team tasks. The schema-based ASyMTRe architecture has successfully solved the problem of automatically synthesizing task solutions based on robot capabilities. However, it does not include a learning ability. Nothing is learned from past experience; therefore, each time a new task needs to be assigned to a new team of robots, the search process for a solution starts anew. Furthermore, it is not possible for the robot to develop a new behavior. The complete SB-CoRLA architecture includes off-line learning and online learning processes. For my dissertation, I implemented a schema chunking process within the framework of SB-CoRLA that involves off-line evolutionary learning of partial solutions (also called “chunks”), and online solution search using learned chunks. The chunks are higher level building blocks than the original schemas. They have similar interfaces to the original schemas, and can be used in an extended version of the ASyMTRe online solution searching process. SB-CoRLA can include other learning processes such as an online learning process that uses a combination of exploration and a goal-directed feedback evaluation process to develop new behaviors by modifying and extending existing schemas. The online learning process is planned for future work. The significance of this work is the development of an architecture that enables continuous, constructivist learning by incorporating learning capabilities in a schema-based robot system, thus allowing robot teams to re-use previous task solutions for both existing and new tasks, to build up more abstract schema chunks, as well as to develop new schemas. The schema chunking process can generate solutions in certain situations when the centralized ASyMTRe cannot find solutions in a timely manner. The chunks can be re-used for different applications, hence improving the search efficiency

Coalition Formation For Distributed Constraint Optimization Problems

This dissertation presents our research on coalition formation for Distributed Constraint Optimization Problems (DCOP). In a DCOP, a problem is broken up into many disjoint sub-problems, each controlled by an autonomous agent and together the system of agents have a joint goal of maximizing a global utility function. In particular, we study the use of coalitions for solving distributed k-coloring problems using iterative approximate algorithms, which do not guarantee optimal results, but provide fast and economic solutions in resource constrained environments. The challenge in forming coalitions using iterative approximate algorithms is in identifying constraint dependencies between agents that allow for effective coalitions to form. We first present the Virtual Structure Reduction (VSR) Algorithm and its integration with a modified version of an iterative approximate solver. The VSR algorithm is the first distributed approach for finding structural relationships, called strict frozen pairs, between agents that allows for effective coalition formation. Using coalition structures allows for both more efficient search and higher overall utility in the solutions. Secondly, we relax the assumption of strict frozen pairs and allow coalitions to form under a probabilistic relationship. We identify probabilistic frozen pairs by calculating the propensity between two agents, or the joint probability of two agents in a k-coloring problem having the same value in all satisfiable instances. Using propensity, we form coalitions in sparse graphs where strict frozen pairs may not exist, but there is still benefit to forming coalitions. Lastly, we present a cooperative game theoretic approach where agents search for Nash stable coalitions under the conditions of additively separable and symmetric value functions