Learning for a robot:deep reinforcement learning, imitation learning, transfer learning

Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed

A robot learning method with physiological interface for teleoperation systems

The human operator largely relies on the perception of remote environmental conditions to make timely and correct decisions in a prescribed task when the robot is teleoperated in a remote place. However, due to the unknown and dynamic working environments, the manipulator's performance and efficiency of the human-robot interaction in the tasks may degrade significantly. In this study, a novel method of human-centric interaction, through a physiological interface was presented to capture the information details of the remote operation environments. Simultaneously, in order to relieve workload of the human operator and to improve efficiency of the teleoperation system, an updated regression method was proposed to build up a nonlinear model of demonstration for the prescribed task. Considering that the demonstration data were of various lengths, dynamic time warping algorithm was employed first to synchronize the data over time before proceeding with other steps. The novelty of this method lies in the fact that both the task-specific information and the muscle parameters from the human operator have been taken into account in a single task; therefore, a more natural and safer interaction between the human and the robot could be achieved. The feasibility of the proposed method was demonstrated by experimental results

Command and Control Systems for Search and Rescue Robots

The novel application of unmanned systems in the domain of humanitarian Search and Rescue (SAR) operations has created a need to develop specific multi-Robot Command and Control (RC2) systems. This societal application of robotics requires human-robot interfaces for controlling a large fleet of heterogeneous robots deployed in multiple domains of operation (ground, aerial and marine). This chapter provides an overview of the Command, Control and Intelligence (C2I) system developed within the scope of Integrated Components for Assisted Rescue and Unmanned Search operations (ICARUS). The life cycle of the system begins with a description of use cases and the deployment scenarios in collaboration with SAR teams as end-users. This is followed by an illustration of the system design and architecture, core technologies used in implementing the C2I, iterative integration phases with field deployments for evaluating and improving the system. The main subcomponents consist of a central Mission Planning and Coordination System (MPCS), field Robot Command and Control (RC2) subsystems with a portable force-feedback exoskeleton interface for robot arm tele-manipulation and field mobile devices. The distribution of these C2I subsystems with their communication links for unmanned SAR operations is described in detail. Field demonstrations of the C2I system with SAR personnel assisted by unmanned systems provide an outlook for implementing such systems into mainstream SAR operations in the future

Switched Kinematic and Force Control for Lower-Limb Motorized Exoskeletons and Functional Electrical Stimulation

Millions of people experience movement deficits from neurological conditions (NCs) that impair their walking ability and leg function. Exercise-based rehabilitation procedures have shown the potential to facilitate neurological reorganization and functional recovery. Lower-limb powered exoskeletons and motorized ergometers have been combined with functional electrical stimulation (FES) to provide repetitive movement, partially reduce the burden of therapists, improve range of motion, and induce therapeutic benefits. FES evokes artificial muscles contractions and can improve muscle mass and strength, and bone density in people with NCs. Stationary cycling is recommended for individuals who cannot perform load-bearing activities or have increased risks of falling. Cycling has been demonstrated to impart physiological and cardiovascular benefits. Motorized FES-cycling combines an electric motor and electrical stimulation of lower-limb muscles to facilitate coordinated, long-duration exercise, while mitigating the inherent muscle fatigue due to FES. Lower-limb exoskeletons coupled with FES, also called neuroprostheses or hybrid exoskeletons, can facilitate continuous, repetitive motion to improve gait function and build muscle capacity. The human-robot interaction during rehabilitative cycling and walking yield a mix of discrete effects (i.e., foot impact, input switching to engage lower-limb muscles and electric motors, etc.) and continuous nonlinear, uncertain, time-varying dynamics. Switching control is necessary to allocate the control inputs to lower-limb muscle groups and electric motors involved during assisted cycling and walking. Kinematic tracking has been the primary control objective for devices that combine FES and electric motors. However, there are force interactions between the machine and the human during cycling and walking that motivate the design of torque-based controllers (i.e., exploit torque or force feedback) to shape the leg dynamics through controlling joint kinematics and kinetics. Technical challenges exist to develop closed-loop feedback control strategies that integrate kinematic and force feedback in the presence of switching and discontinuous effects. The motivation in this dissertation is to design, analyze and implement switching controllers for assisted cycling and walking leveraging kinematic and force feedback while guaranteeing the stability of the human-robot closed-loop system. In Chapter 1, the motivation to design closed-loop controllers for motorized FES-cycling and powered exoskeletons is described. A survey of closed-loop kinematic and force feedback control methods is also introduced related to the tracking objectives presented in the subsequent chapters of the dissertation. In Chapter 2, the dynamics models for walking and assisted cycling are described. First, a bipedal walking system model with switched dynamics is introduced to control a powered lower-limb exoskeleton. Then, a stationary FES-cycling model with nonlinear dynamics and switched control inputs is introduced based on published literature. The muscle stimulation pattern is defined based on the kinematic effectiveness of the rider, which depends on the crank angle. The experimental setup for lower-limb exoskeleton and FES-cycling are described. In Chapter 3, a hierarchical control strategy is developed to interface a cable-driven lower-limb exoskeleton. A two-layer control system is developed to adjust cable tensions and apply torque about the knee joint using a pair of electric motors that provide knee flexion and extension. The control design is segregated into a joint-level control loop and a low-level loop using feedback of the angular positions of the electric motors to mitigate cable slacking. A Lyapunov-based stability analysis is developed to ensure exponential tracking for both control objectives. Moreover, an average dwell time analysis computes an upper bound on the number of motor switches to preserve exponential tracking. Preliminary experimental results in an able-bodied individual are depicted. The developed control strategy is extended and applied to the control of both knee and hip joints in Chapter 4 for treadmill walking. In Chapter 4, a cable-driven lower-limb exoskeleton is integrated with FES for treadmill walking at a constant speed. A nonlinear robust controller is used to activate the quadriceps and hamstrings muscle groups via FES to achieve kinematic tracking about the knee joint. Moreover, electric motors adjust the knee joint stiffness throughout the gait cycle using an integral torque feedback controller. A Lyapunov-based stability analysis is developed to ensure exponential tracking of the kinematic and torque closed-loop error systems, while guaranteeing that the control input signals remain bounded. The developed controllers were tested in real-time walking experiments on a treadmill in three able-bodied individuals at two gait speeds. The experimental results demonstrate the feasibility of coupling a cable-driven exoskeleton with FES for treadmill walking using a switching-based control strategy and exploiting both kinematic and force feedback. In Chapter 5, input-output data is exploited using a finite-time algorithm to estimate the target desired torque leveraging an estimate of the active torque produced by muscles via FES. The convergence rate of the finite-time algorithm can be adjusted by tuning selectable parameters. To achieve cadence and torque tracking for FES-cycling, nonlinear robust tracking controllers are designed for muscles and motor. A Lyapunov-based stability analysis is developed to ensure exponential tracking of the closed-loop cadence error system and global uniformly ultimate bounded (GUUB) torque tracking. A discrete-time Lyapunov-based stability analysis leveraging a recent tool for finite-time systems is developed to ensure convergence and guarantee that the finite-time algorithm is Holder continuous. The developed tracking controllers for the muscles and electric motor and finite-time algorithm to compute the desired torque are implemented in real-time during cycling experiments in seven able-bodied individuals. Multiple cycling trials are implemented with different gain parameters of the finite-time torque algorithm to compare tracking performance for all participants. Chapter 6 highlights the contributions of the developed control methods and provides recommendations for future research extensions

Robotics 2010

Without a doubt, robotics has made an incredible progress over the last decades. The vision of developing, designing and creating technical systems that help humans to achieve hard and complex tasks, has intelligently led to an incredible variety of solutions. There are barely technical fields that could exhibit more interdisciplinary interconnections like robotics. This fact is generated by highly complex challenges imposed by robotic systems, especially the requirement on intelligent and autonomous operation. This book tries to give an insight into the evolutionary process that takes place in robotics. It provides articles covering a wide range of this exciting area. The progress of technical challenges and concepts may illuminate the relationship between developments that seem to be completely different at first sight. The robotics remains an exciting scientific and engineering field. The community looks optimistically ahead and also looks forward for the future challenges and new development

Doctor of Philosophy

dissertationMost humans have difficulty performing precision tasks, such as writing and painting, without additional physical support(s) to help steady or offload their arm's weight. To alleviate this problem, various passive and active devices have been developed. However, such devices often have a small workspace and lack scalable gravity compensation throughout the workspace and/or diversity in their applications. This dissertation describes the development of a Spatial Active Handrest (SAHR), a large-workspace manipulation aid, to offload the weight of the user's arm and increase user's accuracy over a large three-dimensional workspace. This device has four degrees-of-freedom and allows the user to perform dexterous tasks within a large workspace that matches the workspace of a human arm when performing daily tasks. Users can move this device to a desired position and orientation using force or position inputs, or a combination of both. The SAHR converts the given input(s) to desired velocit

Doctor of Philosophy

dissertationHumans generally have difficulty performing precision tasks with their unsupported hands. To compensate for this difficulty, people often seek to support or rest their hand and arm on a fixed surface. However, when the precision task needs to be performed over a workspace larger than what can be reached from a fixed position, a fixed support is no longer useful. This dissertation describes the development of the Active Handrest, a device that expands its user's dexterous workspace by providing ergonomic support and precise repositioning motions over a large workspace. The prototype Active Handrest is a planar computer-controlled support for the user's hand and arm. The device can be controlled through force input from the user, position input from a grasped tool, or a combination of inputs. The control algorithm of the Active Handrest converts the input(s) into device motions through admittance control where the device's desired velocity is calculated proportionally to the input force or its equivalent. A robotic 2-axis admittance device was constructed as the initial Planar Active Handrest, or PAHR, prototype. Experiments were conducted to optimize the device's control input strategies. Large workspace shape tracing experiments were used to compare the PAHR to unsupported, fixed support, and passive moveable support conditions. The Active Handrest was found to reduce task error and provide better speedaccuracy performance. Next, virtual fixture strategies were explored for the device. From the options considered, a virtual spring fixture strategy was chosen based on its effectiveness. An experiment was conducted to compare the PAHR with its virtual fixture strategy to traditional virtual fixture techniques for a grasped stylus. Virtual fixtures implemented on the Active Handrest were found to be as effective as fixtures implemented on a grasped tool. Finally, a higher degree-of-freedom Enhanced Planar Active Handrest, or E-PAHR, was constructed to provide support for large workspace precision tasks while more closely following the planar motions of the human arm. Experiments were conducted to investigate appropriate control strategies and device utility. The E-PAHR was found to provide a skill level equal to that of the PAHR with reduced user force input and lower perceived exertion

Robotic exoskeletons: A perspective for the rehabilitation of arm coordination in stroke patients

Upper-limb impairment after stroke is caused by weakness, loss of individual joint control, spasticity, and abnormal synergies. Upper-limb movement frequently involves abnormal, stereotyped, and fixed synergies, likely related to the increased use of sub-cortical networks following the stroke. The flexible coordination of the shoulder and elbow joints is also disrupted. New methods for motor learning, based on the stimulation of activity- dependent neural plasticity have been developed. These include robots that can adaptively assist active movements and generate many movement repetitions. However, most of these robots only control the movement of the hand in space. The aim of the present text is to analyze the potential of robotic exoskeletons to specifically rehabilitate joint motion and particularly inter-joint coordination. First, a review of studies on upper-limb coordination in stroke patients is presented and the potential for recovery of coordination is examined. Second, issues relating to the mechanical design of exoskeletons and the transmission of constraints between the robotic and human limbs are discussed. The third section considers the development of different methods to control exoskeletons: existing rehabilitation devices and approaches to the control and rehabilitation of joint coordinations are then reviewed, along with preliminary clinical results available. Finally, perspectives and future strategies for the design of control mechanisms for rehabilitation exoskeletons are discussed