Algorithms for Modular Self-reconfigurable Robots: Decision Making, Planning, and Learning

Modular self-reconfigurable robots (MSRs) are composed of multiple robotic modules which can change their connections with each other to take different shapes, commonly known as configurations. Forming different configurations helps the MSR to accomplish different types of tasks in different environments. In this dissertation, we study three different problems in MSRs: partitioning of modules, configuration formation planning and locomotion learning, and we propose algorithmic solutions to solve these problems. Partitioning of modules is a decision-making problem for MSRs where each module decides which partition or team of modules it should be in. To find the best set of partitions is a NP-complete problem. We propose game theory based both centralized and distributed solutions to solve this problem. Once the modules know which set of modules they should team-up with, they self-aggregate to form a specific shaped configuration, known as the configuration formation planning problem. Modules can be either singletons or connected in smaller configurations from which they need to form the target configuration. The configuration formation problem is difficult as multiple modules may select the same location in the target configuration to move to which might result in occlusion and consequently failure of the configuration formation process. On the other hand, if the modules are already in connected configurations in the beginning, then it would be beneficial to preserve those initial configurations for placing them into the target configuration as disconnections and re-connections are costly operations. We propose solutions based on an auction-like algorithm and (sub) graph-isomorphism technique to solve the configuration formation problem. Once the configuration is built, the MSR needs to move towards its goal location as a whole configuration for completing its task. If the configuration’s shape and size is not known a priori, then planning its locomotion is a difficult task as it needs to learn the locomotion pattern in dynamic time – the problem is known as adaptive locomotion learning. We have proposed reinforcement learning based fault-tolerant solutions for locomotion learning by MSRs

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

The Viability of Domain Constrained Coalition Formation for Robotic Collectives

Applications, such as military and disaster response, can benefit from robotic collectives' ability to perform multiple cooperative tasks (e.g., surveillance, damage assessments) efficiently across a large spatial area. Coalition formation algorithms can potentially facilitate collective robots' assignment to appropriate task teams; however, most coalition formation algorithms were designed for smaller multiple robot systems (i.e., 2-50 robots). Collectives' scale and domain-relevant constraints (i.e., distribution, near real-time, minimal communication) make coalition formation more challenging. This manuscript identifies the challenges inherent to designing coalition formation algorithms for very large collectives (e.g., 1000 robots). A survey of multiple robot coalition formation algorithms finds that most are unable to transfer directly to collectives, due to the identified system differences; however, auctions and hedonic games may be the most transferable. A simulation-based evaluation of three auction and hedonic game algorithms, applied to homogeneous and heterogeneous collectives, demonstrates that there are collective compositions for which no existing algorithm is viable; however, the experimental results and literature survey suggest paths forward.Comment: 46 pages, 9 figures, Swarm Intelligence (under review

Enhancing cooperation in wireless networks using different concepts of game theory

PhDOptimizing radio resource within a network and across cooperating heterogeneous networks is the focus of this thesis. Cooperation in a multi-network environment is tackled by investigating network selection mechanisms. These play an important role in ensuring quality of service for users in a multi-network environment. Churning of mobile users from one service provider to another is already common when people change contracts and in a heterogeneous communication environment, where mobile users have freedom to choose the best wireless service-real time selection is expected to become common feature. This real time selection impacts both the technical and the economic aspects of wireless network operations. Next generation wireless networks will enable a dynamic environment whereby the nodes of the same or even different network operator can interact and cooperate to improve their performance. Cooperation has emerged as a novel communication paradigm that can yield tremendous performance gains from the physical layer all the way up to the application layer. Game theory and in particular coalitional game theory is a highly suited mathematical tool for modelling cooperation between wireless networks and is investigated in this thesis. In this thesis, the churning behaviour of wireless service users is modelled by using evolutionary game theory in the context of WLAN access points and WiMAX networks. This approach illustrates how to improve the user perceived QoS in heterogeneous networks using a two-layered optimization. The top layer views the problem of prediction of the network that would be chosen by a user where the criteria are offered bit rate, price, mobility support and reputation. At the second level, conditional on the strategies chosen by the users, the network provider hypothetically, reconfigures the network, subject to the network constraints of bandwidth and acceptable SNR and optimizes the network coverage to support users who would otherwise not be serviced adequately. This forms an iterative cycle until a solution that optimizes the user satisfaction subject to the adjustments that the network provider can make to mitigate the binding constraints, is found and applied to the real network. The evolutionary equilibrium, which is used to 3 compute the average number of users choosing each wireless service, is taken as the solution. This thesis also proposes a fair and practical cooperation framework in which the base stations belonging to the same network provider cooperate, to serve each other‘s customers. How this cooperation can potentially increase their aggregate payoffs through efficient utilization of resources is shown for the case of dynamic frequency allocation. This cooperation framework needs to intelligently determine the cooperating partner and provide a rational basis for sharing aggregate payoff between the cooperative partners for the stability of the coalition. The optimum cooperation strategy, which involves the allocations of the channels to mobile customers, can be obtained as solutions of linear programming optimizations

Sequences of coalition structures in multi-agent systems applied to disaster response

Die Koalitionsbildung ist ein interessantes Thema im Bereich der Multiagentensysteme aufgrund von Herausforderungen bei der praktischen Anwendung, sowie der Komplexität der Berechnung von Lösungen des Problems. Eine Koalition ist ein kurzlebiger Zusammenschluss von Agenten, die ein gemeinsames Ziel verfolgen. Gleichzeitig bietet die kooperative Spieltheorie mit Koalitionen einen formalen Mechanismus zur Analyse von Gruppen aus verschiedenen Akteuren. Daher wird das Problem als Characteristic-Function Game (CFG) modelliert. Dessen Ergebnis sind Aufteilungen einer Menge von Agenten in Koalitionen, sogenannte Koalitionsstrukturen. Allerdings lassen sich nicht alle praktisch auftretenden Probleme effizient mit einer einzigen Koalitionsstruktur lösen. Beispielsweise kann es erforderlich sein, eine Hierarchie von Gruppen zu bilden, in der dann eine Koalitionsstruktur pro Ebene benötigt wird. In der vorliegenden Arbeit werden voneinander abhängige Probleme der Koalitionsbildung untersucht. Insbesondere wird der Schwerpunkt auf die gegenseitige Abhängigkeit von Lösungen (also Koalitionsstrukturen), die aus individuellen Spielen resultieren, gelegt. Angesichts des Mangels an wissenschaftlichen Arbeiten zu diesem Thema wird das Sequential Characteristic-Function Game (SCFG) vorgeschlagen, um die Beziehung zwischen aufeinanderfolgenden Koalitionsstrukturen als Folge von CFGs zu modellieren. Dieses neue Spiel wird erweitert, um spezifische Beschränkungen für jedes CFG in der Spielsequenz zu ermöglichen. Darüber hinaus wird gezeigt, dass das zugrunde liegende SCFG-Problem PSPACE-vollständig ist. Es werden ein exakter Algorithmus zur Berechnung von Lösungen von SCFG-Instanzen, sowie zwei heuristische Algorithmen vorgeschlagen. Die letzte Herausforderung der vorliegenden Arbeit ist die Modellierung eines Katastrophenhilfseinsatzes, bei dem das Einsatzleitsystem (engl. Incident Command System) verwendet wird, mithilfe der vorgeschlagenen Techniken und Algorithmen.Coalition formation has long been an interesting topic of research in Multi-Agent Systems, either for its practical applications or complexity issues. A coalition is commonly understood as a short-lived and goal-directed structure, in which the agents join forces to achieve a goal. Cooperative game theory has been used as a formal mechanism to analyse the problem of grouping agents into coalitions. The problem is then modelled by a Characteristic-Function Game (CFG) in which the outcome is a coalition structure: a partition of agents into coalitions. However, not all problems can be efficiently solved using a single coalition structure. For instance, one might be interested in a group hierarchy in which a coalition structure per level is required. In this thesis, we investigate coalition formation problems that are interdependent. In particular, we focus on the interdependence among solutions (i.e., coalition structures) produced by each game individually. Given the lack of work on this topic, we propose a novel game named Sequential Characteristic-Function Game (SCFG), which aims to model the relationships between subsequent coalition structures in a sequence of CFGs. We approach the resulting problem under both theoretical and practical perspectives. We extend the proposed game to allow fine-grained constraints being induced over each CFG in the sequence. Also, we show that the underlying SCFG problem is PSPACE-complete. From an algorithmic viewpoint, we propose an exact algorithm based on dynamic programming, as well as two heuristic algorithms to compute solutions for SCFG instances. We show that there exists a trade-off in choosing one algorithm over the others. Moreover, we model a disaster response operation that employs the incident command system framework, and we show how one can apply our proposed framework and algorithms to solve such an interesting problem

SB-CoRLA: Schema-Based Constructivist Robot Learning Architecture

This dissertation explores schema-based robot learning. I developed SB-CoRLA (Schema- Based, Constructivist Robot Learning Architecture) to address the issue of constructivist robot learning in a schema-based robot system. The SB-CoRLA architecture extends the previously developed ASyMTRe (Automated Synthesis of Multi-team member Task solutions through software Reconfiguration) architecture to enable constructivist learning for multi-robot team tasks. The schema-based ASyMTRe architecture has successfully solved the problem of automatically synthesizing task solutions based on robot capabilities. However, it does not include a learning ability. Nothing is learned from past experience; therefore, each time a new task needs to be assigned to a new team of robots, the search process for a solution starts anew. Furthermore, it is not possible for the robot to develop a new behavior. The complete SB-CoRLA architecture includes off-line learning and online learning processes. For my dissertation, I implemented a schema chunking process within the framework of SB-CoRLA that involves off-line evolutionary learning of partial solutions (also called “chunks”), and online solution search using learned chunks. The chunks are higher level building blocks than the original schemas. They have similar interfaces to the original schemas, and can be used in an extended version of the ASyMTRe online solution searching process. SB-CoRLA can include other learning processes such as an online learning process that uses a combination of exploration and a goal-directed feedback evaluation process to develop new behaviors by modifying and extending existing schemas. The online learning process is planned for future work. The significance of this work is the development of an architecture that enables continuous, constructivist learning by incorporating learning capabilities in a schema-based robot system, thus allowing robot teams to re-use previous task solutions for both existing and new tasks, to build up more abstract schema chunks, as well as to develop new schemas. The schema chunking process can generate solutions in certain situations when the centralized ASyMTRe cannot find solutions in a timely manner. The chunks can be re-used for different applications, hence improving the search efficiency

Ascending auctions: some impossibility results and their resolutions with final price discounts

When bidders are not substitutes, we show that there is no standard ascend-ing auction that implements a bidder-optimal competitive equilibrium under truthful bidding. Such an impossibility holds also in environments where the Vickrey payoff vector is a competitive equilibrium payoff and is thus stronger than de Vries, Schummer and Vohra s [On ascending Vickrey auctions for het-erogeneous objects, J. Econ. Theory, 132, 95-118] impossibility result with regards to the Vickrey payoff vector under general valuations. Similarly to Mishra and Parkes [Ascending price Vickrey auctions for general valuations, J. Econ. Theory, 132, 335-366], the impossibility can be circumvented by giving price discounts to the bidders from the final vector of prices. Nevertheless, the similarity is misleading: the solution we propose satisfies a minimality infor-mation revelation property that fails to be satisfied in any ascending auction that implements the Vickrey payoffs for general valuations. We investigate related issues when strictly positive increments have to be used under general continuous valuations.Lorsque les enchérisseurs ne sont pas substituts, nous montrons qu'il n'existe pas de mécanisme d'enchères ascendantes qui implémente un équilibre concurrentiel qui soit optimal pour les enchérisseurs. Un tel résultat d'impossibilité reste vrai dans les environnements où les payements de Vickrey sont concurrentiels et est donc plus fort que le résultat d'impossibilité de De Vries, Schummer et Vohra [On ascending Vickrey auctions for heterogeneous objects, J. Econ. Theory, 132, 95-118] relatif à l'implémentation des payements de Vickrey sans restrictions sur les valuations. De la même manière que dans Mishra et Parkes [Ascending price Vickrey auctions for general valuations, J. Econ. Theory, 132, 335-366], l'impossibilité est levée si l'on autorise une phase de réduction des prix à la fin de l'enchère. La similarité est trompeuse : la solution que l'on propose satisfait une propriété de "minimalité" relativement à la révélation des préférences des enchérisseurs, une propriété qui ne peut être satisfaite dans aucune des enchères qui implémente les payements de Vickrey. Nous analysons aussi la robustesse de tels mécanismes à la présence d'incréments

The AI4Citizen pilot: Pipelining AI-based technologies to support school-work alternation programmes

The School-Work Alternation (SWA) programme was developed (under a European Commission call) to bridge the gaps and establish a well-tuned partnership between education and the job market. This work details the development of the AI4Citizen pilot, an AI software suite designed to support the SWA programme. The AI4Citizen pilot, developed within the H2020 AI4EU project, offers AI tools to automate and enhance the current SWA process. At the same time, the AI4Citizen pilot offers novel tools to support the complex problem of allocating student teams to internship programs, promoting collaborative learning and teamwork skills acquisition. Notably, the AI4Citizen pilot corresponds to a pipeline of AI tools, integrating existing and novel technologies. Our exhaustive empirical analysis confirms that the AI4Citizen pilot can alleviate the difficulties of current processes in the SWA, and therefore it is ready for real-world deployment