134 research outputs found
Reset-free Trial-and-Error Learning for Robot Damage Recovery
The high probability of hardware failures prevents many advanced robots
(e.g., legged robots) from being confidently deployed in real-world situations
(e.g., post-disaster rescue). Instead of attempting to diagnose the failures,
robots could adapt by trial-and-error in order to be able to complete their
tasks. In this situation, damage recovery can be seen as a Reinforcement
Learning (RL) problem. However, the best RL algorithms for robotics require the
robot and the environment to be reset to an initial state after each episode,
that is, the robot is not learning autonomously. In addition, most of the RL
methods for robotics do not scale well with complex robots (e.g., walking
robots) and either cannot be used at all or take too long to converge to a
solution (e.g., hours of learning). In this paper, we introduce a novel
learning algorithm called "Reset-free Trial-and-Error" (RTE) that (1) breaks
the complexity by pre-generating hundreds of possible behaviors with a dynamics
simulator of the intact robot, and (2) allows complex robots to quickly recover
from damage while completing their tasks and taking the environment into
account. We evaluate our algorithm on a simulated wheeled robot, a simulated
six-legged robot, and a real six-legged walking robot that are damaged in
several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and
whose objective is to reach a sequence of targets in an arena. Our experiments
show that the robots can recover most of their locomotion abilities in an
environment with obstacles, and without any human intervention.Comment: 18 pages, 16 figures, 3 tables, 6 pseudocodes/algorithms, video at
https://youtu.be/IqtyHFrb3BU, code at
https://github.com/resibots/chatzilygeroudis_2018_rt
Evolving a Behavioral Repertoire for a Walking Robot
Numerous algorithms have been proposed to allow legged robots to learn to
walk. However, the vast majority of these algorithms is devised to learn to
walk in a straight line, which is not sufficient to accomplish any real-world
mission. Here we introduce the Transferability-based Behavioral Repertoire
Evolution algorithm (TBR-Evolution), a novel evolutionary algorithm that
simultaneously discovers several hundreds of simple walking controllers, one
for each possible direction. By taking advantage of solutions that are usually
discarded by evolutionary processes, TBR-Evolution is substantially faster than
independently evolving each controller. Our technique relies on two methods:
(1) novelty search with local competition, which searches for both
high-performing and diverse solutions, and (2) the transferability approach,
which com-bines simulations and real tests to evolve controllers for a physical
robot. We evaluate this new technique on a hexapod robot. Results show that
with only a few dozen short experiments performed on the robot, the algorithm
learns a repertoire of con-trollers that allows the robot to reach every point
in its reachable space. Overall, TBR-Evolution opens a new kind of learning
algorithm that simultaneously optimizes all the achievable behaviors of a
robot.Comment: 33 pages; Evolutionary Computation Journal 201
Legged Robots for Object Manipulation: A Review
Legged robots can have a unique role in manipulating objects in dynamic,
human-centric, or otherwise inaccessible environments. Although most legged
robotics research to date typically focuses on traversing these challenging
environments, many legged platform demonstrations have also included "moving an
object" as a way of doing tangible work. Legged robots can be designed to
manipulate a particular type of object (e.g., a cardboard box, a soccer ball,
or a larger piece of furniture), by themselves or collaboratively. The
objective of this review is to collect and learn from these examples, to both
organize the work done so far in the community and highlight interesting open
avenues for future work. This review categorizes existing works into four main
manipulation methods: object interactions without grasping, manipulation with
walking legs, dedicated non-locomotive arms, and legged teams. Each method has
different design and autonomy features, which are illustrated by available
examples in the literature. Based on a few simplifying assumptions, we further
provide quantitative comparisons for the range of possible relative sizes of
the manipulated object with respect to the robot. Taken together, these
examples suggest new directions for research in legged robot manipulation, such
as multifunctional limbs, terrain modeling, or learning-based control, to
support a number of new deployments in challenging indoor/outdoor scenarios in
warehouses/construction sites, preserved natural areas, and especially for home
robotics.Comment: Preprint of the paper submitted to Frontiers in Mechanical
Engineerin
Legged locomotion over irregular terrains: State of the art of human and robot performance
Legged robotic technologies have moved out of the lab to operate in real environments, characterized by a wide variety of unpredictable irregularities and disturbances, all this in close proximity with humans. Demonstrating the ability of current robots to move robustly and reliably in these conditions is becoming essential to prove their safe operation. Here, we report an in-depth literature review aimed at verifying the existence of common or agreed protocols and metrics to test the performance of legged system in realistic environments. We primarily focused on three types of robotic technologies, i.e., hexapods, quadrupeds and bipeds. We also included a comprehensive overview on human locomotion studies, being it often considered the gold standard for performance, and one of the most important sources of bioinspiration for legged machines. We discovered that very few papers have rigorously studied robotic locomotion under irregular terrain conditions. On the contrary, numerous studies have addressed this problem on human gait, being nonetheless of highly heterogeneous nature in terms of experimental design. This lack of agreed methodology makes it challenging for the community to properly assess, compare and predict the performance of existing legged systems in real environments. On the one hand, this work provides a library of methods, metrics and experimental protocols, with a critical analysis on the limitations of the current approaches and future promising directions. On the other hand, it demonstrates the existence of an important lack of benchmarks in the literature, and the possibility of bridging different disciplines, e.g., the human and robotic, towards the definition of standardized procedure that will boost not only the scientific development of better bioinspired solutions, but also their market uptake
A survey on policy search algorithms for learning robot controllers in a handful of trials
Most policy search algorithms require thousands of training episodes to find
an effective policy, which is often infeasible with a physical robot. This
survey article focuses on the extreme other end of the spectrum: how can a
robot adapt with only a handful of trials (a dozen) and a few minutes? By
analogy with the word "big-data", we refer to this challenge as "micro-data
reinforcement learning". We show that a first strategy is to leverage prior
knowledge on the policy structure (e.g., dynamic movement primitives), on the
policy parameters (e.g., demonstrations), or on the dynamics (e.g.,
simulators). A second strategy is to create data-driven surrogate models of the
expected reward (e.g., Bayesian optimization) or the dynamical model (e.g.,
model-based policy search), so that the policy optimizer queries the model
instead of the real system. Overall, all successful micro-data algorithms
combine these two strategies by varying the kind of model and prior knowledge.
The current scientific challenges essentially revolve around scaling up to
complex robots (e.g., humanoids), designing generic priors, and optimizing the
computing time.Comment: 21 pages, 3 figures, 4 algorithms, accepted at IEEE Transactions on
Robotic
Low-cost Printable Robots in Education
The final publication is available at Springer via http://dx.doi.org/10.1007/s10846-015-0199-xThe wider availability of 3D printing has enabled small printable robots (or printbots) to be incorporated directly into engineering courses. Printbots can be used in many ways to enhance lifelong learning skills, strengthen understanding and foster teamwork and collaboration. The experiences outlined in this paper were used in our teaching during the last academic year, although much of the methodology and many of the activities have been used and developed over the past 8 years. They include project based assignments carried out by multidisciplinary and multicultural teams, a number of theoretical and practical classroom and laboratory activities all aimed at familiarizing students with fundamental concepts, programming and simulation, and which now form part of our regular robotics courses, and some brief descriptions of how printable robots are being used by students carrying out final projects for Bachelor and Master degrees. The online resources show many of these activities in action.Armesto Ángel, L.; Fuentes-Durá, P.; Perry, DR. (2016). Low-cost Printable Robots in Education. Journal of Intelligent and Robotic Systems. 81(1):5-24. doi:10.1007/s10846-015-0199-xS524811Criteria for accrediting engineering programs (Unknown Month 2015, 2014). http://www.abet.org/eac-criteria-2014-2015Board, N.S.: Moving forward to improve engineering education (2007). http://www.nsf.gov/pubs/2007/nsb07122/nsb07122.pdfCampion, G., Bastin, G., d’Andréa Novel, B.: Structural properties and classification of kinematic and dynamic models of wheeled mobile robots. IEEE Trans. Robot. Autom. 12(1), 47–62 (1996)Carberry, A.R., Lee, H.-S., Ohland, M.W.: Measuring engineering design self-efficacy. J. Eng. Educ. 99(1), 71–79 (2010)Castro. A.: Robotic arm with 6 dof (2012). http://www.thingiverse.com/thing:30163Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G.A., Burgard, W., Kavraki, L.E., Thrun, S.: Principles of Robot Motion: Theory, Algorithms, and Implementations. MIT Press, Cambridge MA (2005)d’Andréa Novel, B., Campion, G., Bastin, G.: Control of nonholonomic wheeled mobile robots by state feedback linearization. Int. J. Robot. Res. 14(6), 543–559 (1995)Denavit, J., Hartenberg, R.S.: A kinematic notation for lower-pair mechanisms based on matrices. Trans. ASME J. Appl. Mech 22(2), 215–221 (1955)Dowdall. J.: Rofi robot five (2012). http://www.projectbiped.com/prototypes/rofiEliot, M., Howard, P., Nouwens, F., Stojcevski, A., Mann, L., Prpic, J., Gabb, R., Venkatesan, S., Kolmos, A.: Developing a conceptual model for the effective assessment of individual student learning in team-based subjects. Australas. J. Eng. Educ. 18(1), 105–112 (2012)Fox, D., Burgard, W., Thrun, S.: The dynamic window approach to collision avoidance. Robot. Autom. Mag. IEEE 4(1), 23–33 (1997)Fuentes-Dura, P., Armesto, L., Perry, D.: Multidisciplinary projects: Critical points and perceptions in valladolid in innovation and quality in engineering education. In: Innovation and Quality in Engineering Education, pp 315–331 (2012)Fuentes-Dura, P., Cazorla, M.P., Molina, M.G., Perry, D.: European project semester: Good practices for competence acquisition. In: Valencia Global, pp 165– 172 (2014)González, J., Barrientos, A., Prieto-Moreno, A., de Frutos, M.A.: Miniskybot 2 (2012). http://www.iearobotics.com/wiki/index.php?Miniskybot_2Gonzalez-Gomez, J., Valero-Gomez, A., Prieto-Moreno, A., Abderrahim, M.: A new open source 3d-printable mobile robotic platform for education. In: Rckert, U., Joaquin, S., Felix, W. (eds.) Advances in Autonomous Mini Robots, pp 49–62. Springer, Berlin Heidelberg (2012)Gonzlez, J., Wagenaar, R. (eds.): Tuning Educational Structures in Europe University of Deusto and Groningen. Deusto (2003)Heinrich, E., Bhattacharya, M., Rayudu, R.: Preparation for lifelong learning using eportfolios. Eur. J. Eng. Educ. 32(6), 653–663 (2007)Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. The Int. J. Robot. Res. 5(1), 90–98 (1986)Krassman, J.: Quadcopter hummingbird ii (2013). http://www.thingiverse.com/thing:167721Langevin, G.: Inmoov (2012). http://www.inmoov.frMadox: ecanum wheel rover 2 (2011). http://www.madox.net/blog/2011/01/24/mecanum-wheel-rover-2Miles, M.B., Analysis, A.M.: Huberman. Qualitative Data: An Expanded Sourcebook. SAGE Publications (1994)Minguez, J., Montano, L.: Nearness diagram (nd) navigation: Collision avoidance in troublesome scenarios. IEEE Trans. Robot. Autom. 20, 2004 (2004)Olalla: Caterpillator v1.1 (2011). http://www.thingiverse.com/thing:8559Ollero, A.: Robótica. Manipuladores y robots móviles Marcombo, S.A. Barcelona (2001)Price, M.: Hf08 hexapod robot (2012). http://www.heliumfrog.com/hf08robot/hf08blog.htmlRawat, K., Massiha, G.: A hands-on laboratory based approach to undergraduate robotics education. In: Proceedings of 2004 IEEE International Conference on Robotics and Automation 2, pp 1370–1374 (2004)Robotics, C.: Virtual experimentation robotic platform (v-rep) (2013). www.coppeliarobotics.comScott, B.: Principles of problem and project based learning the aalborg model. Aalbord University (2010)Teichler, U., Schonburg, H.: editors. Comparative Perspectives on Higher Education and Graduate Employment and Work Experiences from Twelve Countries. Kluwer Pub. (2004)Ulrich, I., Borenstein, J.: Vfh+: reliable obstacle avoidance for fast mobile robots. In: Robotics and Automation, 1998. Proceedings, volume 2, pp 1572–1577 (1998)Verner, I., Waks, S., Kolberg, E.: Educational robotics An insight into systems engineering. Eur. J. Eng. Educ. 24(2), 201–212 (1999)C.y.A. Vicerrectorado de Estudios: Dimensiones competenciales upv (2013). http://www.upv.es/contenidos/ICEP/info/DimensionesCompetenciales.pdfWampler, C.W.: Manipulator inverse kinematic solutions based on vector formulations and damped least squares methods. IEEE Trans. Syst. Man, Cybern. 16(1), 93–101 (1986)Weinberg, J., Yu, X.: Robotics in education Low-cost platforms for teaching integrated systems. Robot. Autom. Mag. IEEE 10(2), 4–6 (2003
- …