591 research outputs found
Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network
In this paper, we present TeachNet, a novel neural network architecture for
intuitive and markerless vision-based teleoperation of dexterous robotic hands.
Robot joint angles are directly generated from depth images of the human hand
that produce visually similar robot hand poses in an end-to-end fashion. The
special structure of TeachNet, combined with a consistency loss function,
handles the differences in appearance and anatomy between human and robotic
hands. A synchronized human-robot training set is generated from an existing
dataset of labeled depth images of the human hand and simulated depth images of
a robotic hand. The final training set includes 400K pairwise depth images and
joint angles of a Shadow C6 robotic hand. The network evaluation results verify
the superiority of TeachNet, especially regarding the high-precision condition.
Imitation experiments and grasp tasks teleoperated by novice users demonstrate
that TeachNet is more reliable and faster than the state-of-the-art
vision-based teleoperation method.Comment: Accepted to ICRA 2019. Shuang Li and Xiaojian Ma contributed equally
to this wor
Exploiting the robot kinematic redundancy for emotion conveyance to humans as a lower priority task
Current approaches do not allow robots to execute a task and simultaneously convey emotions to users using their body motions. This paper explores the capabilities of the Jacobian null space of a humanoid robot to convey emotions. A task priority formulation has been implemented in a Pepper robot which allows the specification of a primary task (waving gesture, transportation of an object, etc.) and exploits the kinematic redundancy of the robot to convey emotions to humans as a lower priority task. The emotions, defined by Mehrabian as points in the pleasure–arousal–dominance space, generate intermediate motion features (jerkiness, activity and gaze) that carry the emotional information. A map from this features to the joints of the robot is presented. A user study has been conducted in which emotional motions have been shown to 30 participants. The results show that happiness and sadness are very well conveyed to the user, calm is moderately well conveyed, and fear is not well conveyed. An analysis on the dependencies between the motion features and the emotions perceived by the participants shows that activity correlates positively with arousal, jerkiness is not perceived by the user, and gaze conveys dominance when activity is low. The results indicate a strong influence of the most energetic motions of the emotional task and point out new directions for further research. Overall, the results show that the null space approach can be regarded as a promising mean to convey emotions as a lower priority task.Postprint (author's final draft
Model-Based Environmental Visual Perception for Humanoid Robots
The visual perception of a robot should answer two fundamental questions: What? and Where? In order to properly and efficiently reply to these questions, it is essential to establish a bidirectional coupling between the external stimuli and the internal representations. This coupling links the physical world with the inner abstraction models by sensor transformation, recognition, matching and optimization algorithms. The objective of this PhD is to establish this sensor-model coupling
A Probabilistic Framework for Learning Kinematic Models of Articulated Objects
Robots operating in domestic environments generally need to interact with
articulated objects, such as doors, cabinets, dishwashers or fridges. In this
work, we present a novel, probabilistic framework for modeling articulated
objects as kinematic graphs. Vertices in this graph correspond to object parts,
while edges between them model their kinematic relationship. In particular, we
present a set of parametric and non-parametric edge models and how they can
robustly be estimated from noisy pose observations. We furthermore describe how
to estimate the kinematic structure and how to use the learned kinematic models
for pose prediction and for robotic manipulation tasks. We finally present how
the learned models can be generalized to new and previously unseen objects. In
various experiments using real robots with different camera systems as well as
in simulation, we show that our approach is valid, accurate and efficient.
Further, we demonstrate that our approach has a broad set of applications, in
particular for the emerging fields of mobile manipulation and service robotics
Pose Estimation from a Single Depth Image for Arbitrary Kinematic Skeletons
We present a method for estimating pose information from a single depth image
given an arbitrary kinematic structure without prior training. For an arbitrary
skeleton and depth image, an evolutionary algorithm is used to find the optimal
kinematic configuration to explain the observed image. Results show that our
approach can correctly estimate poses of 39 and 78 degree-of-freedom models
from a single depth image, even in cases of significant self-occlusion.Comment: 2 pages, 2 figures, RGB-D workshop in Robotics: Science and Systems
(RSS 2011
Context-Aware System Synthesis, Task Assignment, and Routing
The design and organization of complex robotic systems traditionally requires
laborious trial-and-error processes to ensure both hardware and software
components are correctly connected with the resources necessary for
computation. This paper presents a novel generalization of the quadratic
assignment and routing problem, introducing formalisms for selecting components
and interconnections to synthesize a complete system capable of providing some
user-defined functionality. By introducing mission context, functional
requirements, and modularity directly into the assignment problem, we derive a
solution where components are automatically selected and then organized into an
optimal hardware and software interconnection structure, all while respecting
restrictions on component viability and required functionality. The ability to
generate \emph{complete} functional systems directly from individual components
reduces manual design effort by allowing for a guided exploration of the design
space. Additionally, our formulation increases resiliency by quantifying
resource margins and enabling adaptation of system structure in response to
changing environments, hardware or software failure. The proposed formulation
is cast as an integer linear program which is provably -hard. Two
case studies are developed and analyzed to highlight the expressiveness and
complexity of problems that can be addressed by this approach: the first
explores the iterative development of a ground-based search-and-rescue robot in
a variety of mission contexts, while the second explores the large-scale,
complex design of a humanoid disaster robot for the DARPA Robotics Challenge.
Numerical simulations quantify real world performance and demonstrate tractable
time complexity for the scale of problems encountered in many modern robotic
systems.Comment: 17 pages, 10 figures, Submitted to Transactions in Robotic
On the Calibration of Active Binocular and RGBD Vision Systems for Dual-Arm Robots
This paper describes a camera and hand-eye
calibration methodology for integrating an active binocular
robot head within a dual-arm robot. For this purpose, we
derive the forward kinematic model of our active robot head
and describe our methodology for calibrating and integrating
our robot head. This rigid calibration provides a closedform
hand-to-eye solution. We then present an approach for
updating dynamically camera external parameters for optimal
3D reconstruction that are the foundation for robotic tasks such
as grasping and manipulating rigid and deformable objects. We
show from experimental results that our robot head achieves
an overall sub millimetre accuracy of less than 0.3 millimetres
while recovering the 3D structure of a scene. In addition, we
report a comparative study between current RGBD cameras
and our active stereo head within two dual-arm robotic testbeds
that demonstrates the accuracy and portability of our proposed
methodology
Learning Extended Body Schemas from Visual Keypoints for Object Manipulation
Humans have impressive generalization capabilities when it comes to
manipulating objects and tools in completely novel environments. These
capabilities are, at least partially, a result of humans having internal models
of their bodies and any grasped object. How to learn such body schemas for
robots remains an open problem. In this work, we develop an approach that can
extend a robot's kinematic model when grasping an object from visual latent
representations. Our framework comprises two components: 1) a structured
keypoint detector, which fuses proprioception and vision to predict visual key
points on an object; 2) Learning an adaptation of the kinematic chain by
regressing virtual joints from the predicted key points. Our evaluation shows
that our approach learns to consistently predict visual keypoints on objects,
and can easily adapt a kinematic chain to the object grasped in various
configurations, from a few seconds of data. Finally we show that this extended
kinematic chain lends itself for object manipulation tasks such as placing a
grasped object
Recommended from our members
Analysis and synthesis of bipedal humanoid movement : a physical simulation approach
textAdvances in graphics and robotics have increased the importance of tools for synthesizing humanoid movements to control animated characters and physical robots. There is also an increasing need for analyzing human movements for clinical diagnosis and rehabilitation. Existing tools can be expensive, inefficient, or difficult to use. Using simulated physics and motion capture to develop an interactive virtual reality environment, we capture natural human movements in response to controlled stimuli. This research then applies insights into the mathematics underlying physics simulation to adapt the physics solver to support many important tasks involved in analyzing and synthesizing humanoid movement. These tasks include fitting an articulated physical model to motion capture data, modifying the model pose to achieve a desired configuration (inverse kinematics), inferring internal torques consistent with changing pose data (inverse dynamics), and transferring a movement from one model to another model (retargeting). The result is a powerful and intuitive process for analyzing and synthesizing movement in a single unified framework.Computer Science
Simultaneous Localization and Mapping (SLAM) on NAO
Simultaneous Localization and Mapping (SLAM) is a navigation and mapping method used by autonomous robots and moving vehicles. SLAM is mainly concerned with the problem of building a map in an unknown environment and concurrently navigating through the environment using the map. Localization is of utmost importance to allow the robot to keep track of its position with respect to the environment and the common use of odometry proves to be unreliable. SLAM has been proposed as a solution by previous research to provide more accurate localization and mapping on robots. This project involves the implementation of the SLAM algorithm in the humanoid robot NAO by Aldebaran Robotics. The SLAM technique will be implemented using vision from the single camera attached to the robot to map and localize the position of NAO in the environment. The result details the attempt to implement specifically the chosen algorithm, 1-Point RANSAC Inverse Depth EKF Monocular SLAM by Dr Javier Civera on the robot NAO. The algorithm is shown to perform well for smooth motions but on the humanoid NAO, the sudden changes in motion produces undesirable results.This study on SLAM will be useful as this technique can be widely used to allow mobile robots to map and navigate in areas which are deemed unsafe for humans
- …