4,097 research outputs found

    Distributed reinforcement learning for self-reconfiguring modular robots

    Get PDF
    Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 101-106).In this thesis, we study distributed reinforcement learning in the context of automating the design of decentralized control for groups of cooperating, coupled robots. Specifically, we develop a framework and algorithms for automatically generating distributed controllers for self-reconfiguring modular robots using reinforcement learning. The promise of self-reconfiguring modular robots is that of robustness, adaptability and versatility. Yet most state-of-the-art distributed controllers are laboriously handcrafted and task-specific, due to the inherent complexities of distributed, local-only control. In this thesis, we propose and develop a framework for using reinforcement learning for automatic generation of such controllers. The approach is profitable because reinforcement learning methods search for good behaviors during the lifetime of the learning agent, and are therefore applicable to online adaptation as well as automatic controller design. However, we must overcome the challenges due to the fundamental partial observability inherent in a distributed system such as a self reconfiguring modular robot. We use a family of policy search methods that we adapt to our distributed problem. The outcome of a local search is always influenced by the search space dimensionality, its starting point, and the amount and quality of available exploration through experience.(cont) We undertake a systematic study of the effects that certain robot and task parameters, such as the number of modules, presence of exploration constraints, availability of nearest-neighbor communications, and partial behavioral knowledge from previous experience, have on the speed and reliability of learning through policy search in self-reconfiguring modular robots. In the process, we develop novel algorithmic variations and compact search space representations for learning in our domain, which we test experimentally on a number of tasks. This thesis is an empirical study of reinforcement learning in a simulated lattice based self-reconfiguring modular robot domain. However, our results contribute to the broader understanding of automatic generation of group control and design of distributed reinforcement learning algorithms.by Paulina Varshavskaya.Ph.D

    Universal Reconfiguration of Facet-Connected Modular Robots by Pivots: The O(1) Musketeers

    Get PDF
    We present the first universal reconfiguration algorithm for transforming a modular robot between any two facet-connected square-grid configurations using pivot moves. More precisely, we show that five extra "helper" modules ("musketeers") suffice to reconfigure the remaining n modules between any two given configurations. Our algorithm uses O(n^2) pivot moves, which is worst-case optimal. Previous reconfiguration algorithms either require less restrictive "sliding" moves, do not preserve facet-connectivity, or for the setting we consider, could only handle a small subset of configurations defined by a local forbidden pattern. Configurations with the forbidden pattern do have disconnected reconfiguration graphs (discrete configuration spaces), and indeed we show that they can have an exponential number of connected components. But forbidding the local pattern throughout the configuration is far from necessary, as we show that just a constant number of added modules (placed to be freely reconfigurable) suffice for universal reconfigurability. We also classify three different models of natural pivot moves that preserve facet-connectivity, and show separations between these models

    Reconfiguration of 3D Crystalline Robots Using O(log n) Parallel Moves

    Full text link
    We consider the theoretical model of Crystalline robots, which have been introduced and prototyped by the robotics community. These robots consist of independently manipulable unit-square atoms that can extend/contract arms on each side and attach/detach from neighbors. These operations suffice to reconfigure between any two given (connected) shapes. The worst-case number of sequential moves required to transform one connected configuration to another is known to be Theta(n). However, in principle, atoms can all move simultaneously. We develop a parallel algorithm for reconfiguration that runs in only O(log n) parallel steps, although the total number of operations increases slightly to Theta(nlogn). The result is the first (theoretically) almost-instantaneous universally reconfigurable robot built from simple units.Comment: 21 pages, 10 figure

    Modular Self-Reconfigurable Robot Systems

    Get PDF
    The field of modular self-reconfigurable robotic systems addresses the design, fabrication, motion planning, and control of autonomous kinematic machines with variable morphology. Modular self-reconfigurable systems have the promise of making significant technological advances to the field of robotics in general. Their promise of high versatility, high value, and high robustness may lead to a radical change in automation. Currently, a number of researchers have been addressing many of the challenges. While some progress has been made, it is clear that many challenges still exist. By illustrating several of the outstanding issues as grand challenges that have been collaboratively written by a large number of researchers in this field, this article has shown several of the key directions for the future of this growing fiel

    Collective Construction of 2D Block Structures with Holes

    Full text link
    In this paper we present algorithms for collective construction systems in which a large number of autonomous mobile robots trans- port modular building elements to construct a desired structure. We focus on building block structures subject to some physical constraints that restrict the order in which the blocks may be attached to the structure. Specifically, we determine a partial ordering on the blocks such that if they are attached in accordance with this ordering, then (i) the structure is a single, connected piece at all intermediate stages of construction, and (ii) no block is attached between two other previously attached blocks, since such a space is too narrow for a robot to maneuver a block into it. Previous work has consider this problem for building 2D structures without holes. Here we extend this work to 2D structures with holes. We accomplish this by modeling the problem as a graph orientation problem and describe an O(n^2) algorithm for solving it. We also describe how this partial ordering may be used in a distributed fashion by the robots to coordinate their actions during the building process.Comment: 13 pages, 3 figure

    A new meta-module for efficient reconfiguration of hinged-units modular robots

    Get PDF
    We present a robust and compact meta-module for edge-hinged modular robot units such as M-TRAN, SuperBot, SMORES, UBot, PolyBot and CKBot, as well as for central-point-hinged ones such as Molecubes and Roombots. Thanks to the rotational degrees of freedom of these units, the novel meta-module is able to expand and contract, as to double/halve its length in each dimension. Moreover, for a large class of edge-hinged robots the proposed meta-module also performs the scrunch/relax and transfer operations required by any tunneling-based reconfiguration strategy, such as those designed for Crystalline and Telecube robots. These results make it possible to apply efficient geometric reconfiguration algorithms to this type of robots. We prove the size of this new meta-module to be optimal. Its robustness and performance substantially improve over previous results.Peer ReviewedPostprint (author's final draft
    • …
    corecore