134 research outputs found

    Reset-free Trial-and-Error Learning for Robot Damage Recovery

    Get PDF
    The high probability of hardware failures prevents many advanced robots (e.g., legged robots) from being confidently deployed in real-world situations (e.g., post-disaster rescue). Instead of attempting to diagnose the failures, robots could adapt by trial-and-error in order to be able to complete their tasks. In this situation, damage recovery can be seen as a Reinforcement Learning (RL) problem. However, the best RL algorithms for robotics require the robot and the environment to be reset to an initial state after each episode, that is, the robot is not learning autonomously. In addition, most of the RL methods for robotics do not scale well with complex robots (e.g., walking robots) and either cannot be used at all or take too long to converge to a solution (e.g., hours of learning). In this paper, we introduce a novel learning algorithm called "Reset-free Trial-and-Error" (RTE) that (1) breaks the complexity by pre-generating hundreds of possible behaviors with a dynamics simulator of the intact robot, and (2) allows complex robots to quickly recover from damage while completing their tasks and taking the environment into account. We evaluate our algorithm on a simulated wheeled robot, a simulated six-legged robot, and a real six-legged walking robot that are damaged in several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and whose objective is to reach a sequence of targets in an arena. Our experiments show that the robots can recover most of their locomotion abilities in an environment with obstacles, and without any human intervention.Comment: 18 pages, 16 figures, 3 tables, 6 pseudocodes/algorithms, video at https://youtu.be/IqtyHFrb3BU, code at https://github.com/resibots/chatzilygeroudis_2018_rt

    Evolving a Behavioral Repertoire for a Walking Robot

    Full text link
    Numerous algorithms have been proposed to allow legged robots to learn to walk. However, the vast majority of these algorithms is devised to learn to walk in a straight line, which is not sufficient to accomplish any real-world mission. Here we introduce the Transferability-based Behavioral Repertoire Evolution algorithm (TBR-Evolution), a novel evolutionary algorithm that simultaneously discovers several hundreds of simple walking controllers, one for each possible direction. By taking advantage of solutions that are usually discarded by evolutionary processes, TBR-Evolution is substantially faster than independently evolving each controller. Our technique relies on two methods: (1) novelty search with local competition, which searches for both high-performing and diverse solutions, and (2) the transferability approach, which com-bines simulations and real tests to evolve controllers for a physical robot. We evaluate this new technique on a hexapod robot. Results show that with only a few dozen short experiments performed on the robot, the algorithm learns a repertoire of con-trollers that allows the robot to reach every point in its reachable space. Overall, TBR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.Comment: 33 pages; Evolutionary Computation Journal 201

    Legged Robots for Object Manipulation: A Review

    Get PDF
    Legged robots can have a unique role in manipulating objects in dynamic, human-centric, or otherwise inaccessible environments. Although most legged robotics research to date typically focuses on traversing these challenging environments, many legged platform demonstrations have also included "moving an object" as a way of doing tangible work. Legged robots can be designed to manipulate a particular type of object (e.g., a cardboard box, a soccer ball, or a larger piece of furniture), by themselves or collaboratively. The objective of this review is to collect and learn from these examples, to both organize the work done so far in the community and highlight interesting open avenues for future work. This review categorizes existing works into four main manipulation methods: object interactions without grasping, manipulation with walking legs, dedicated non-locomotive arms, and legged teams. Each method has different design and autonomy features, which are illustrated by available examples in the literature. Based on a few simplifying assumptions, we further provide quantitative comparisons for the range of possible relative sizes of the manipulated object with respect to the robot. Taken together, these examples suggest new directions for research in legged robot manipulation, such as multifunctional limbs, terrain modeling, or learning-based control, to support a number of new deployments in challenging indoor/outdoor scenarios in warehouses/construction sites, preserved natural areas, and especially for home robotics.Comment: Preprint of the paper submitted to Frontiers in Mechanical Engineerin

    Intelligent approaches in locomotion - a review

    Get PDF

    Legged locomotion over irregular terrains: State of the art of human and robot performance

    Get PDF
    Legged robotic technologies have moved out of the lab to operate in real environments, characterized by a wide variety of unpredictable irregularities and disturbances, all this in close proximity with humans. Demonstrating the ability of current robots to move robustly and reliably in these conditions is becoming essential to prove their safe operation. Here, we report an in-depth literature review aimed at verifying the existence of common or agreed protocols and metrics to test the performance of legged system in realistic environments. We primarily focused on three types of robotic technologies, i.e., hexapods, quadrupeds and bipeds. We also included a comprehensive overview on human locomotion studies, being it often considered the gold standard for performance, and one of the most important sources of bioinspiration for legged machines. We discovered that very few papers have rigorously studied robotic locomotion under irregular terrain conditions. On the contrary, numerous studies have addressed this problem on human gait, being nonetheless of highly heterogeneous nature in terms of experimental design. This lack of agreed methodology makes it challenging for the community to properly assess, compare and predict the performance of existing legged systems in real environments. On the one hand, this work provides a library of methods, metrics and experimental protocols, with a critical analysis on the limitations of the current approaches and future promising directions. On the other hand, it demonstrates the existence of an important lack of benchmarks in the literature, and the possibility of bridging different disciplines, e.g., the human and robotic, towards the definition of standardized procedure that will boost not only the scientific development of better bioinspired solutions, but also their market uptake

    A survey on policy search algorithms for learning robot controllers in a handful of trials

    Get PDF
    Most policy search algorithms require thousands of training episodes to find an effective policy, which is often infeasible with a physical robot. This survey article focuses on the extreme other end of the spectrum: how can a robot adapt with only a handful of trials (a dozen) and a few minutes? By analogy with the word "big-data", we refer to this challenge as "micro-data reinforcement learning". We show that a first strategy is to leverage prior knowledge on the policy structure (e.g., dynamic movement primitives), on the policy parameters (e.g., demonstrations), or on the dynamics (e.g., simulators). A second strategy is to create data-driven surrogate models of the expected reward (e.g., Bayesian optimization) or the dynamical model (e.g., model-based policy search), so that the policy optimizer queries the model instead of the real system. Overall, all successful micro-data algorithms combine these two strategies by varying the kind of model and prior knowledge. The current scientific challenges essentially revolve around scaling up to complex robots (e.g., humanoids), designing generic priors, and optimizing the computing time.Comment: 21 pages, 3 figures, 4 algorithms, accepted at IEEE Transactions on Robotic

    Low-cost Printable Robots in Education

    Full text link
    The final publication is available at Springer via http://dx.doi.org/10.1007/s10846-015-0199-xThe wider availability of 3D printing has enabled small printable robots (or printbots) to be incorporated directly into engineering courses. Printbots can be used in many ways to enhance lifelong learning skills, strengthen understanding and foster teamwork and collaboration. The experiences outlined in this paper were used in our teaching during the last academic year, although much of the methodology and many of the activities have been used and developed over the past 8 years. They include project based assignments carried out by multidisciplinary and multicultural teams, a number of theoretical and practical classroom and laboratory activities all aimed at familiarizing students with fundamental concepts, programming and simulation, and which now form part of our regular robotics courses, and some brief descriptions of how printable robots are being used by students carrying out final projects for Bachelor and Master degrees. The online resources show many of these activities in action.Armesto Ángel, L.; Fuentes-Durá, P.; Perry, DR. (2016). Low-cost Printable Robots in Education. Journal of Intelligent and Robotic Systems. 81(1):5-24. doi:10.1007/s10846-015-0199-xS524811Criteria for accrediting engineering programs (Unknown Month 2015, 2014). http://www.abet.org/eac-criteria-2014-2015Board, N.S.: Moving forward to improve engineering education (2007). http://www.nsf.gov/pubs/2007/nsb07122/nsb07122.pdfCampion, G., Bastin, G., d’Andréa Novel, B.: Structural properties and classification of kinematic and dynamic models of wheeled mobile robots. IEEE Trans. Robot. Autom. 12(1), 47–62 (1996)Carberry, A.R., Lee, H.-S., Ohland, M.W.: Measuring engineering design self-efficacy. J. Eng. Educ. 99(1), 71–79 (2010)Castro. A.: Robotic arm with 6 dof (2012). http://www.thingiverse.com/thing:30163Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G.A., Burgard, W., Kavraki, L.E., Thrun, S.: Principles of Robot Motion: Theory, Algorithms, and Implementations. MIT Press, Cambridge MA (2005)d’Andréa Novel, B., Campion, G., Bastin, G.: Control of nonholonomic wheeled mobile robots by state feedback linearization. Int. J. Robot. Res. 14(6), 543–559 (1995)Denavit, J., Hartenberg, R.S.: A kinematic notation for lower-pair mechanisms based on matrices. Trans. ASME J. Appl. Mech 22(2), 215–221 (1955)Dowdall. J.: Rofi robot five (2012). http://www.projectbiped.com/prototypes/rofiEliot, M., Howard, P., Nouwens, F., Stojcevski, A., Mann, L., Prpic, J., Gabb, R., Venkatesan, S., Kolmos, A.: Developing a conceptual model for the effective assessment of individual student learning in team-based subjects. Australas. J. Eng. Educ. 18(1), 105–112 (2012)Fox, D., Burgard, W., Thrun, S.: The dynamic window approach to collision avoidance. Robot. Autom. Mag. IEEE 4(1), 23–33 (1997)Fuentes-Dura, P., Armesto, L., Perry, D.: Multidisciplinary projects: Critical points and perceptions in valladolid in innovation and quality in engineering education. In: Innovation and Quality in Engineering Education, pp 315–331 (2012)Fuentes-Dura, P., Cazorla, M.P., Molina, M.G., Perry, D.: European project semester: Good practices for competence acquisition. In: Valencia Global, pp 165– 172 (2014)González, J., Barrientos, A., Prieto-Moreno, A., de Frutos, M.A.: Miniskybot 2 (2012). http://www.iearobotics.com/wiki/index.php?Miniskybot_2Gonzalez-Gomez, J., Valero-Gomez, A., Prieto-Moreno, A., Abderrahim, M.: A new open source 3d-printable mobile robotic platform for education. In: Rckert, U., Joaquin, S., Felix, W. (eds.) Advances in Autonomous Mini Robots, pp 49–62. Springer, Berlin Heidelberg (2012)Gonzlez, J., Wagenaar, R. (eds.): Tuning Educational Structures in Europe University of Deusto and Groningen. Deusto (2003)Heinrich, E., Bhattacharya, M., Rayudu, R.: Preparation for lifelong learning using eportfolios. Eur. J. Eng. Educ. 32(6), 653–663 (2007)Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. The Int. J. Robot. Res. 5(1), 90–98 (1986)Krassman, J.: Quadcopter hummingbird ii (2013). http://www.thingiverse.com/thing:167721Langevin, G.: Inmoov (2012). http://www.inmoov.frMadox: ecanum wheel rover 2 (2011). http://www.madox.net/blog/2011/01/24/mecanum-wheel-rover-2Miles, M.B., Analysis, A.M.: Huberman. Qualitative Data: An Expanded Sourcebook. SAGE Publications (1994)Minguez, J., Montano, L.: Nearness diagram (nd) navigation: Collision avoidance in troublesome scenarios. IEEE Trans. Robot. Autom. 20, 2004 (2004)Olalla: Caterpillator v1.1 (2011). http://www.thingiverse.com/thing:8559Ollero, A.: Robótica. Manipuladores y robots móviles Marcombo, S.A. Barcelona (2001)Price, M.: Hf08 hexapod robot (2012). http://www.heliumfrog.com/hf08robot/hf08blog.htmlRawat, K., Massiha, G.: A hands-on laboratory based approach to undergraduate robotics education. In: Proceedings of 2004 IEEE International Conference on Robotics and Automation 2, pp 1370–1374 (2004)Robotics, C.: Virtual experimentation robotic platform (v-rep) (2013). www.coppeliarobotics.comScott, B.: Principles of problem and project based learning the aalborg model. Aalbord University (2010)Teichler, U., Schonburg, H.: editors. Comparative Perspectives on Higher Education and Graduate Employment and Work Experiences from Twelve Countries. Kluwer Pub. (2004)Ulrich, I., Borenstein, J.: Vfh+: reliable obstacle avoidance for fast mobile robots. In: Robotics and Automation, 1998. Proceedings, volume 2, pp 1572–1577 (1998)Verner, I., Waks, S., Kolberg, E.: Educational robotics An insight into systems engineering. Eur. J. Eng. Educ. 24(2), 201–212 (1999)C.y.A. Vicerrectorado de Estudios: Dimensiones competenciales upv (2013). http://www.upv.es/contenidos/ICEP/info/DimensionesCompetenciales.pdfWampler, C.W.: Manipulator inverse kinematic solutions based on vector formulations and damped least squares methods. IEEE Trans. Syst. Man, Cybern. 16(1), 93–101 (1986)Weinberg, J., Yu, X.: Robotics in education Low-cost platforms for teaching integrated systems. Robot. Autom. Mag. IEEE 10(2), 4–6 (2003
    corecore