Dynamic movement primitives based cloud robotic skill learning for point and non-point obstacle avoidance

Purpose: Dynamic movement primitives (DMPs) is a general robotic skill learning from demonstration method, but it is usually used for single robotic manipulation. For cloud-based robotic skill learning, the authors consider trajectories/skills changed by the environment, rebuild the DMPs model and propose a new DMPs-based skill learning framework removing the influence of the changing environment. Design/methodology/approach: The authors proposed methods for two obstacle avoidance scenes: point obstacle and non-point obstacle. For the case with point obstacles, an accelerating term is added to the original DMPs function. The unknown parameters in this term are estimated by interactive identification and fitting step of the forcing function. Then a pure skill despising the influence of obstacles is achieved. Using identified parameters, the skill can be applied to new tasks with obstacles. For the non-point obstacle case, a space matching method is proposed by building a matching function from the universal space without obstacle to the space condensed by obstacles. Then the original trajectory will change along with transformation of the space to get a general trajectory for the new environment. Findings: The proposed two methods are certified by two experiments, one of which is taken based on Omni joystick to record operator’s manipulation motions. Results show that the learned skills allow robots to execute tasks such as autonomous assembling in a new environment. Originality/value: This is a new innovation for DMPs-based cloud robotic skill learning from multi-scene tasks and generalizing new skills following the changes of the environment

DMPs-based skill learning for redundant dual-arm robotic synchronized cooperative manipulation

Dual-arm robot manipulation is applicable to many domains, such as industrial, medical, and home service scenes. Learning from demonstrations (LfD) is a highly effective paradigm for robotic learning, where a robot learns from human actions directly and can be used autonomously for new tasks, avoiding the complicated analytical calculation for motion programming. However, the learned skills are not easy to generalize to new cases where special constraints such as varying relative distance limitation of robotic end effectors for human-like cooperative manipulations exist. In this paper, we propose a dynamic movement primitives (DMPs) based skills learning framework for redundant dual-arm robots. The method, with a coupling acceleration term to the DMPs function, is inspired by the transient performance control of Barrier Lyapunov Functions (BLFs). The additional coupling acceleration term is calculated based on the constant joint distance and varying relative distance limitations of end effectors for object approaching actions. In addition, we integrate the generated actions in joint space and the solution for a redundant dual-arm robot to complete a human-like manipulation. Simulations undertaken in Matlab and Gazebo environments certify the effectiveness of the proposed method

Neural Network iLQR: A New Reinforcement Learning Architecture

As a notable machine learning paradigm, the research efforts in the context of reinforcement learning have certainly progressed leaps and bounds. When compared with reinforcement learning methods with the given system model, the methodology of the reinforcement learning architecture based on the unknown model generally exhibits significantly broader universality and applicability. In this work, a new reinforcement learning architecture is developed and presented without the requirement of any prior knowledge of the system model, which is termed as an approach of a "neural network iterative linear quadratic regulator (NNiLQR)". Depending solely on measurement data, this method yields a completely new non-parametric routine for the establishment of the optimal policy (without the necessity of system modeling) through iterative refinements of the neural network system. Rather importantly, this approach significantly outperforms the classical iterative linear quadratic regulator (iLQR) method in terms of the given objective function because of the innovative utilization of further exploration in the methodology. As clearly indicated from the results attained in two illustrative examples, these significant merits of the NNiLQR method are demonstrated rather evidently.Comment: 13 pages, 9 figure

Robot skill learning system of multi-space fusion based on dynamic movement primitives and adaptive neural network control

This article develops a robot skill learning system with multi-space fusion, simultaneously considering motion/stiffness generation and trajectory tracking. To begin with, surface electromyography (sEMG) signals from the human arm is captured based on the MYO armband to estimate endpoint stiffness. Gaussian Process Regression (GPR) is combined with dynamic movement primitive (DMP) to extract more skills features from multi-demonstrations. Then, the traditional DMP formulation is improved based on the Riemannian metric to encode the robot's quaternions with non-Euclidean properties. Furthermore, an adaptive neural network (NN)-based finite-time admittance controller is designed to track the trajectory generated by the motion model and to reflect the learned stiffness characteristics. In this controller, a radial basis function neural network (RBFNN) is employed to compensate for the uncertainty of the robot dynamics. Finally, experimental validation is conducted using the ROKAE collaborative robot, confirming the effectiveness of the proposed approach. In summary, the presented framework is suitable for human-robot skill transfer method that require simultaneous consideration of position and stiffness in Euclidean space, as well as orientation on Riemannian manifolds

A multimodal human-robot sign language interaction framework applied in social robots

Deaf-mutes face many difficulties in daily interactions with hearing people through spoken language. Sign language is an important way of expression and communication for deaf-mutes. Therefore, breaking the communication barrier between the deaf-mute and hearing communities is significant for facilitating their integration into society. To help them integrate into social life better, we propose a multimodal Chinese sign language (CSL) gesture interaction framework based on social robots. The CSL gesture information including both static and dynamic gestures is captured from two different modal sensors. A wearable Myo armband and a Leap Motion sensor are used to collect human arm surface electromyography (sEMG) signals and hand 3D vectors, respectively. Two modalities of gesture datasets are preprocessed and fused to improve the recognition accuracy and to reduce the processing time cost of the network before sending it to the classifier. Since the input datasets of the proposed framework are temporal sequence gestures, the long-short term memory recurrent neural network is used to classify these input sequences. Comparative experiments are performed on an NAO robot to test our method. Moreover, our method can effectively improve CSL gesture recognition accuracy, which has potential applications in a variety of gesture interaction scenarios not only in social robots

A constrained DMPs framework for robot skills learning and generalization from human demonstrations

Dynamical movement primitives (DMPs) model is a useful tool for efficiently robotic learning manipulation skills from human demonstrations and then generalizing these skills to fulfill new tasks. It is improved and applied for the cases with multiple constraints such as having obstacles or relative distance limitation for multi-agent formation. However, the improved DMPs should change additional terms according to the specified constraints of different tasks. In this paper, we will propose a novel DMPs framework facing the constrained conditions for robotic skills generalization. First, we conclude the common characteristics of previous modified DMPs with constraints and propose a general DMPs framework with various classified constraints. Inspired by barrier Lyapunov functions (BLFs), an additional acceleration term of the general model is deduced to compensate tracking errors between the real and desired trajectories with constraints. Furthermore, we prove convergence of the generated path and makes a discussion about advantages of the proposed method compared with existing literature. Finally, we instantiate the novel framework through three experiments: obstacle avoidance in the static and dynamic environment and human-like cooperative manipulation, to certify its effectiveness

A trajectory and force dual-incremental robot skill learning and generalization framework using improved dynamical movement primitives and adaptive neural network control

Due to changes in the environment and errors that occurred during skill initialization, the robot's operational skills should be modified to adapt to new tasks. As such, skills learned by the methods with fixed features, such as the classical Dynamical Movement Primitive (DMP), are difficult to use when the using cases are significantly different from the demonstrations. In this work, we propose an incremental robot skill learning and generalization framework including an incremental DMP (IDMP) for robot trajectory learning and an adaptive neural network (NN) control method, which are incrementally updated to enable robots to adapt to new cases. IDMP uses multi-mapping feature vectors to rebuild the forcing function of DMP, which are extended based on the original feature vector. In order to maintain the original skills and represent skill changes in a new task, the new feature vector consists of three parts with different usages. Therefore, the trajectories are gradually changed by expanding the feature and weight vectors, and all transition states are also easily recovered. Then, an adaptive NN controller with performance constraints is proposed to compensate dynamics errors and changed trajectories after using the IDMP. The new controller is also incrementally updated and can accumulate and reuse the learned knowledge to improve the learning efficiency. Compared with other methods, the proposed framework achieves higher tracking accuracy, realizes incremental skill learning and modification, achieves multiple stylistic skills, and is used for obstacle avoidance with different heights, which are verified in three comparative experiments