5,069 research outputs found
GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
We address the highly challenging problem of real-time 3D hand tracking based
on a monocular RGB-only sequence. Our tracking method combines a convolutional
neural network with a kinematic 3D hand model, such that it generalizes well to
unseen data, is robust to occlusions and varying camera viewpoints, and leads
to anatomically plausible as well as temporally smooth hand motions. For
training our CNN we propose a novel approach for the synthetic generation of
training data that is based on a geometrically consistent image-to-image
translation network. To be more specific, we use a neural network that
translates synthetic images to "real" images, such that the so-generated images
follow the same statistical distribution as real-world hand images. For
training this translation network we combine an adversarial loss and a
cycle-consistency loss with a geometric consistency loss in order to preserve
geometric properties (such as hand pose) during translation. We demonstrate
that our hand tracking system outperforms the current state-of-the-art on
challenging RGB-only footage
Rule Of Thumb: Deep derotation for improved fingertip detection
We investigate a novel global orientation regression approach for articulated
objects using a deep convolutional neural network. This is integrated with an
in-plane image derotation scheme, DeROT, to tackle the problem of per-frame
fingertip detection in depth images. The method reduces the complexity of
learning in the space of articulated poses which is demonstrated by using two
distinct state-of-the-art learning based hand pose estimation methods applied
to fingertip detection. Significant classification improvements are shown over
the baseline implementation. Our framework involves no tracking, kinematic
constraints or explicit prior model of the articulated object in hand. To
support our approach we also describe a new pipeline for high accuracy magnetic
annotation and labeling of objects imaged by a depth camera.Comment: To be published in proceedings of BMVC 201
Efficient Jacobian-Based Inverse Kinematics With Sim-to-Real Transfer of Soft Robots by Learning
This paper presents an efficient learning-based method to solve the inverse
kinematic (IK) problem on soft robots with highly non-linear deformation. The
major challenge of efficiently computing IK for such robots is due to the lack
of analytical formulation for either forward or inverse kinematics. To address
this challenge, we employ neural networks to learn both the mapping function of
forward kinematics and also the Jacobian of this function. As a result,
Jacobian-based iteration can be applied to solve the IK problem. A sim-to-real
training transfer strategy is conducted to make this approach more practical.
We first generate a large number of samples in a simulation environment for
learning both the kinematic and the Jacobian networks of a soft robot design.
Thereafter, a sim-to-real layer of differentiable neurons is employed to map
the results of simulation to the physical hardware, where this sim-to-real
layer can be learned from a very limited number of training samples generated
on the hardware. The effectiveness of our approach has been verified on
pneumatic-driven soft robots for path following and interactive positioning
- …