Algorithms for Modular Self-reconfigurable Robots: Decision Making, Planning, and Learning

Modular self-reconfigurable robots (MSRs) are composed of multiple robotic modules which can change their connections with each other to take different shapes, commonly known as configurations. Forming different configurations helps the MSR to accomplish different types of tasks in different environments. In this dissertation, we study three different problems in MSRs: partitioning of modules, configuration formation planning and locomotion learning, and we propose algorithmic solutions to solve these problems. Partitioning of modules is a decision-making problem for MSRs where each module decides which partition or team of modules it should be in. To find the best set of partitions is a NP-complete problem. We propose game theory based both centralized and distributed solutions to solve this problem. Once the modules know which set of modules they should team-up with, they self-aggregate to form a specific shaped configuration, known as the configuration formation planning problem. Modules can be either singletons or connected in smaller configurations from which they need to form the target configuration. The configuration formation problem is difficult as multiple modules may select the same location in the target configuration to move to which might result in occlusion and consequently failure of the configuration formation process. On the other hand, if the modules are already in connected configurations in the beginning, then it would be beneficial to preserve those initial configurations for placing them into the target configuration as disconnections and re-connections are costly operations. We propose solutions based on an auction-like algorithm and (sub) graph-isomorphism technique to solve the configuration formation problem. Once the configuration is built, the MSR needs to move towards its goal location as a whole configuration for completing its task. If the configuration’s shape and size is not known a priori, then planning its locomotion is a difficult task as it needs to learn the locomotion pattern in dynamic time – the problem is known as adaptive locomotion learning. We have proposed reinforcement learning based fault-tolerant solutions for locomotion learning by MSRs

Dynamic Reconfiguration in Modular Self-Reconfigurable Robots Using Multi-Agent Coalition Games

In this thesis, we consider the problem of autonomous self-reconfiguration by modular self-reconfigurable robots (MSRs). MSRs are composed of small units or modules that can be dynamically configured to form different structures, such as a lattice or a chain. The main problem in maneuvering MSRs is to enable them to autonomously reconfigure their structure depending on the operational conditions in the environment. We first discuss limitations of previous approaches to solve the MSR self-reconfiguration problem. We will then present a novel framework that uses a layered architecture comprising a conventional gait table-based maneuver to move the robot in a fixed configuration, but using a more complex coalition game-based technique for autonomously reconfiguring the robot. We discuss the complexity of solving the reconfiguration problem within the coalition game-based framework and propose a stochastic planning and pruning based approach to solve the coalition-game based MSR reconfiguration problem. We tested our MSR self-reconfiguration algorithm using an accurately simulated model of an MSR called ModRED (Modular Robot for Exploration and Discovery) within the Webots robot simulator. Our results show that using our coalition formation algorithm, MSRs are able to reconfigure efficiently after encountering an obstacle. The average “reward” or efficiency obtained by an MSR also improves by 2-10% while using our coalition formation algorithm as compared to a previously existing multi-agent coalition formation algorithm. To the best of our knowledge, this work represents two novel contributions in the field of modular robots. First, ours is one of the first research techniques that has combined principles from human team formation techniques from the area of computational economics with dynamic self-reconfiguration in modular self-reconfigurable robots. Secondly, the modeling of uncertainty in coalition games using Markov Decision Processes is a novel and previously unexplored problem in the area of coalition formation. Overall, this thesis addresses a challenging research problem at the intersection of artificial intelligence, game theory and robotics and opens up several new directions for further research to improve the control and reconfiguration of modular robots

Self-Reconfiguration Planning in Modular Reconfigurable Robots

MSRs are highly versatile robots that work together to form into different configurations. However, to take advantage of this ability to transform, the MSR must utilize an SRP algorithm to determine what actions to perform to shape itself to reach its goal configuration. An SRP algorithm can be boiled down to a search method through an unexplored graph which we approach with four basic search algorithms to see which algorithm is best when designing an SRP algorithm. To do this we create a general MSR model known as stickbots and use different search algorithms on a variety of SRP problems to test the model. With these tests we can observe how different algorithms are affected by different scenarios. With the data collected using this model we hope to show that certain algorithms are better suited for creating SRP algorithms and our model can be used to test more complex algorithms

Autonomous Operation of a Reconfigurable Multi-Robot System for Planetary Space Missions

Reconfigurable robots can physically merge and form new types of composite systems. This ability leads to additional degrees of freedom for robot operations especially when dynamically composed robotic systems offer capabilities that none of the individual systems have. Research in the area of reconfigurable multi-robot systems has mainly been focused on swarm-based robots and thereby to systems with a high degree of modularity but a heavily restricted set of capabilities. In contrast, this thesis deals with heterogeneous robot teams comprising individually capable robots which are also modular and reconfigurable. In particular, the autonomous application of such reconfigurable multi-robot systems to enhance robotic space exploration missions is investigated. Exploiting the flexibility of a reconfigurable multi-robot system requires an appropriate system model and reasoner. Hence, this thesis introduces a special organisation model. This model accounts for the key characteristics of reconfigurable robots which are constrained by the availability and compatibility of hardware interfaces. A newly introduced mapping function between resource structures and functional properties permits to characterise dynamically created agent compositions. Since a combinatorial challenge lies in the identification of feasible and functionally suitable agents, this thesis further suggests bounding strategies to reason efficiently with composite robotic systems. This thesis proposes a mission planning algorithm which permits to exploit the flexibility of reconfigurable multi-robot systems. The implemented planner builds upon the developed organisation model so that multi-robot missions can be specified by high-level functionality constraints which are resolved to suitable combinations of robots. Furthermore, the planner synchronises robot activities over time and characterises plans according to three objectives: efficacy, efficiency and safety. The plannera s evaluation demonstrates an optimization of an exemplary space mission. This research is based on the parallel development of theoretical concepts and practical solutions while working with three reconfigurable multi-robot teams. The operation of a reconfigurable robotic team comes with practical constraints. Therefore, this thesis composes and evaluates an operational infrastructure which can serve as reference implementation. The identification and combination of applicable state-of-the-art technologies result in a distributed and dynamically extensible communication infrastructure which can maintain the properties of reconfigurable multi-robot systems. Field tests covering semi-autonomous and autonomous operation have been performed to characterise multi-robot missions and validate the autonomous control approach for reconfigurable multi-robot systems. The practical evaluation identified critical constraints and design elements for a successful application of reconfigurable multi-robot systems. Furthermore, the experiments point to improvements for the organisation model. This thesis is a wholistic approach to automate reconfigurable multi-robot systems. It identifies theoretical as well as practical challenges and it suggests effective solutions which permit an exploitation of an increased level of flexibility in future robotics missions

Autonomous Operation of a Reconfigurable Multi-Robot System for Planetary Space Missions

Reconfigurable robots can physically merge and form new types of composite systems. This ability leads to additional degrees of freedom for robot operations especially when dynamically composed robotic systems offer capabilities that none of the individual systems have. Research in the area of reconfigurable multi-robot systems has mainly been focused on swarm-based robots and thereby to systems with a high degree of modularity but a heavily restricted set of capabilities. In contrast, this thesis deals with heterogeneous robot teams comprising individually capable robots which are also modular and reconfigurable. In particular, the autonomous application of such reconfigurable multi-robot systems to enhance robotic space exploration missions is investigated. Exploiting the flexibility of a reconfigurable multi-robot system requires an appropriate system model and reasoner. Hence, this thesis introduces a special organisation model. This model accounts for the key characteristics of reconfigurable robots which are constrained by the availability and compatibility of hardware interfaces. A newly introduced mapping function between resource structures and functional properties permits to characterise dynamically created agent compositions. Since a combinatorial challenge lies in the identification of feasible and functionally suitable agents, this thesis further suggests bounding strategies to reason efficiently with composite robotic systems. This thesis proposes a mission planning algorithm which permits to exploit the flexibility of reconfigurable multi-robot systems. The implemented planner builds upon the developed organisation model so that multi-robot missions can be specified by high-level functionality constraints which are resolved to suitable combinations of robots. Furthermore, the planner synchronises robot activities over time and characterises plans according to three objectives: efficacy, efficiency and safety. The plannera s evaluation demonstrates an optimization of an exemplary space mission. This research is based on the parallel development of theoretical concepts and practical solutions while working with three reconfigurable multi-robot teams. The operation of a reconfigurable robotic team comes with practical constraints. Therefore, this thesis composes and evaluates an operational infrastructure which can serve as reference implementation. The identification and combination of applicable state-of-the-art technologies result in a distributed and dynamically extensible communication infrastructure which can maintain the properties of reconfigurable multi-robot systems. Field tests covering semi-autonomous and autonomous operation have been performed to characterise multi-robot missions and validate the autonomous control approach for reconfigurable multi-robot systems. The practical evaluation identified critical constraints and design elements for a successful application of reconfigurable multi-robot systems. Furthermore, the experiments point to improvements for the organisation model. This thesis is a wholistic approach to automate reconfigurable multi-robot systems. It identifies theoretical as well as practical challenges and it suggests effective solutions which permit an exploitation of an increased level of flexibility in future robotics missions

Algorithms for Graph-Constrained Coalition Formation in the Real World

Coalition formation typically involves the coming together of multiple, heterogeneous, agents to achieve both their individual and collective goals. In this paper, we focus on a special case of coalition formation known as Graph-Constrained Coalition Formation (GCCF) whereby a network connecting the agents constrains the formation of coalitions. We focus on this type of problem given that in many real-world applications, agents may be connected by a communication network or only trust certain peers in their social network. We propose a novel representation of this problem based on the concept of edge contraction, which allows us to model the search space induced by the GCCF problem as a rooted tree. Then, we propose an anytime solution algorithm (CFSS), which is particularly efficient when applied to a general class of characteristic functions called $m+a$ functions. Moreover, we show how CFSS can be efficiently parallelised to solve GCCF using a non-redundant partition of the search space. We benchmark CFSS on both synthetic and realistic scenarios, using a real-world dataset consisting of the energy consumption of a large number of households in the UK. Our results show that, in the best case, the serial version of CFSS is 4 orders of magnitude faster than the state of the art, while the parallel version is 9.44 times faster than the serial version on a 12-core machine. Moreover, CFSS is the first approach to provide anytime approximate solutions with quality guarantees for very large systems of agents (i.e., with more than 2700 agents).Comment: Accepted for publication, cite as "in press

Design of an UAV swarm

This master thesis tries to give an overview on the general aspects involved in the design of an UAV swarm. UAV swarms are continuoulsy gaining popularity amongst researchers and UAV manufacturers, since they allow greater success rates in task accomplishing with reduced times. Appart from this, multiple UAVs cooperating between them opens a new field of missions that can only be carried in this way. All the topics explained within this master thesis will explain all the agents involved in the design of an UAV swarm, from the communication protocols between them, navigation and trajectory analysis and task allocation

A reconfigurable distributed multiagent system optimized for scalability

This thesis proposes a novel solution for optimizing the size and communication overhead of a distributed multiagent system without compromising the performance. The proposed approach addresses the challenges of scalability especially when the multiagent system is large. A modified spectral clustering technique is used to partition a large network into logically related clusters. Agents are assigned to monitor dedicated clusters rather than monitor each device or node. The proposed scalable multiagent system is implemented using JADE (Java Agent Development Environment) for a large power system. The performance of the proposed topology-independent decentralized multiagent system and the scalable multiagent system is compared by comprehensively simulating different fault scenarios. The time taken for reconfiguration, the overall computational complexity, and the communication overhead incurred are computed. The results of these simulations show that the proposed scalable multiagent system uses fewer agents efficiently, makes faster decisions to reconfigure when a fault occurs, and incurs significantly less communication overhead. The proposed scalable multiagent system has been coupled with a scalable reconfiguration algorithm for an electric power system attempting to minimize the number of switch combination explored for reconfiguration. The reconfiguration algorithm reconfigures a power system while maintaining bus voltages within limits specified by constraints

Role Based Hedonic Games

In the hedonic coalition formation game model Roles Based Hedonic Games (RBHG), agents view teams as compositions of available roles. An agent\u27s utility for a partition is based upon which role she fulfills within the coalition and which additional roles are being fulfilled within the coalition. I consider optimization and stability problems for settings with variable power on the part of the central authority and on the part of the agents. I prove several of these problems to be NP-complete or coNP-complete. I introduce heuristic methods for approximating solutions for a variety of these hard problems. I validate heuristics on real-world data scraped from League of Legends games

SB-CoRLA: Schema-Based Constructivist Robot Learning Architecture

This dissertation explores schema-based robot learning. I developed SB-CoRLA (Schema- Based, Constructivist Robot Learning Architecture) to address the issue of constructivist robot learning in a schema-based robot system. The SB-CoRLA architecture extends the previously developed ASyMTRe (Automated Synthesis of Multi-team member Task solutions through software Reconfiguration) architecture to enable constructivist learning for multi-robot team tasks. The schema-based ASyMTRe architecture has successfully solved the problem of automatically synthesizing task solutions based on robot capabilities. However, it does not include a learning ability. Nothing is learned from past experience; therefore, each time a new task needs to be assigned to a new team of robots, the search process for a solution starts anew. Furthermore, it is not possible for the robot to develop a new behavior. The complete SB-CoRLA architecture includes off-line learning and online learning processes. For my dissertation, I implemented a schema chunking process within the framework of SB-CoRLA that involves off-line evolutionary learning of partial solutions (also called “chunks”), and online solution search using learned chunks. The chunks are higher level building blocks than the original schemas. They have similar interfaces to the original schemas, and can be used in an extended version of the ASyMTRe online solution searching process. SB-CoRLA can include other learning processes such as an online learning process that uses a combination of exploration and a goal-directed feedback evaluation process to develop new behaviors by modifying and extending existing schemas. The online learning process is planned for future work. The significance of this work is the development of an architecture that enables continuous, constructivist learning by incorporating learning capabilities in a schema-based robot system, thus allowing robot teams to re-use previous task solutions for both existing and new tasks, to build up more abstract schema chunks, as well as to develop new schemas. The schema chunking process can generate solutions in certain situations when the centralized ASyMTRe cannot find solutions in a timely manner. The chunks can be re-used for different applications, hence improving the search efficiency