Recursive recurrent neural network: A novel model for manipulator control with different levels of physical constraints

Manipulators actuate joints to let end effectors to perform precise path tracking tasks. Recurrent neural network which is described by dynamic models with parallel processing capability, is a powerful tool for kinematic control of manipulators. Due to physical limitations and actuation saturation of manipulator joints, the involvement of joint constraints for kinematic control of manipulators is essential and critical. However, current existing manipulator control methods based on recurrent neural networks mainly handle with limited levels of joint angular constraints, and to the best of our knowledge, methods for kinematic control of manipulators with higher order joint constraints based on recurrent neural networks are not yet reported. In this study, for the first time, a novel recursive recurrent network model is proposed to solve the kinematic control issue for manipulators with different levels of physical constraints, and the proposed recursive recurrent neural network can be formulated as a new manifold system to ensure control solution within all of the joint constraints in different orders. The theoretical analysis shows the stability and the purposed recursive recurrent neural network and its convergence to solution. Simulation results further demonstrate the effectiveness of the proposed method in end-effector path tracking control under different levels of joint constraints based on the Kuka manipulator system. Comparisons with other methods such as the pseudoinverse-based method and conventional recurrent neural network method substantiate the superiority of the proposed method

Skeleton-aided Articulated Motion Generation

This work make the first attempt to generate articulated human motion sequence from a single image. On the one hand, we utilize paired inputs including human skeleton information as motion embedding and a single human image as appearance reference, to generate novel motion frames, based on the conditional GAN infrastructure. On the other hand, a triplet loss is employed to pursue appearance-smoothness between consecutive frames. As the proposed framework is capable of jointly exploiting the image appearance space and articulated/kinematic motion space, it generates realistic articulated motion sequence, in contrast to most previous video generation methods which yield blurred motion effects. We test our model on two human action datasets including KTH and Human3.6M, and the proposed framework generates very promising results on both datasets.Comment: ACM MM 201

Recurrent 3D Pose Sequence Machines

3D human articulated pose recovery from monocular image sequences is very challenging due to the diverse appearances, viewpoints, occlusions, and also the human 3D pose is inherently ambiguous from the monocular imagery. It is thus critical to exploit rich spatial and temporal long-range dependencies among body joints for accurate 3D pose sequence prediction. Existing approaches usually manually design some elaborate prior terms and human body kinematic constraints for capturing structures, which are often insufficient to exploit all intrinsic structures and not scalable for all scenarios. In contrast, this paper presents a Recurrent 3D Pose Sequence Machine(RPSM) to automatically learn the image-dependent structural constraint and sequence-dependent temporal context by using a multi-stage sequential refinement. At each stage, our RPSM is composed of three modules to predict the 3D pose sequences based on the previously learned 2D pose representations and 3D poses: (i) a 2D pose module extracting the image-dependent pose representations, (ii) a 3D pose recurrent module regressing 3D poses and (iii) a feature adaption module serving as a bridge between module (i) and (ii) to enable the representation transformation from 2D to 3D domain. These three modules are then assembled into a sequential prediction framework to refine the predicted poses with multiple recurrent stages. Extensive evaluations on the Human3.6M dataset and HumanEva-I dataset show that our RPSM outperforms all state-of-the-art approaches for 3D pose estimation.Comment: Published in CVPR 201