1,332 research outputs found
Robot Composite Learning and the Nunchaku Flipping Challenge
Advanced motor skills are essential for robots to physically coexist with
humans. Much research on robot dynamics and control has achieved success on
hyper robot motor capabilities, but mostly through heavily case-specific
engineering. Meanwhile, in terms of robot acquiring skills in a ubiquitous
manner, robot learning from human demonstration (LfD) has achieved great
progress, but still has limitations handling dynamic skills and compound
actions. In this paper, we present a composite learning scheme which goes
beyond LfD and integrates robot learning from human definition, demonstration,
and evaluation. The method tackles advanced motor skills that require dynamic
time-critical maneuver, complex contact control, and handling partly soft
partly rigid objects. We also introduce the "nunchaku flipping challenge", an
extreme test that puts hard requirements to all these three aspects. Continued
from our previous presentations, this paper introduces the latest update of the
composite learning scheme and the physical success of the nunchaku flipping
challenge
Development of a Fingertip Glove Equipped with Magnetic Tracking Sensors
In this paper, we present the development of a data glove system based on fingertip tracking techniques. To track the fingertip position and orientation, a sensor module and two generator coils are attached on the fingertip and metacarpal of the corresponding finger. By tracking the fingertip, object manipulation tasks in a virtual environment or teleoperation system can be carried out more precisely, because fingertips are the foremost areas that reach the surface of an object in most of grasping processes. To calculate the bending angles of a finger, we also propose a method of constructing the shape of the finger. Since the coils are installed on the fingertips and metacarpals, there is no contact point between the sensors and finger joints. Hence, the shape of the sensors does not change as the fingers are bending, and both the quality of measurement and the lifetime of the sensors will not decrease in time. For the convenience of using this glove, a simple and efficient calibration process consisting of only one calibration gesture is also provided, so that all required parameters can be determined automatically. So far, the experimental results of the sensors performing linear movement and bending angle measurements are very satisfactory. It reveals that our data glove is available for a man-machine interface
Human to robot hand motion mapping methods: review and classification
In this article, the variety of approaches proposed in literature to address the problem of mapping human to robot hand motions are summarized and discussed. We particularly attempt to organize under macro-categories the great quantity of presented methods, that are often difficult to be seen from a general point of view due to different fields of application, specific use of algorithms, terminology and declared goals of the mappings. Firstly, a brief historical overview is reported, in order to provide a look on the emergence of the human to robot hand mapping problem as a both conceptual and analytical challenge that is still open nowadays. Thereafter, the survey mainly focuses on a classification of modern mapping methods under six categories: direct joint, direct Cartesian, taskoriented, dimensionality reduction based, pose recognition based and hybrid mappings. For each of these categories, the general view that associates the related reported studies is provided, and representative references are highlighted. Finally, a concluding discussion along with the authors’ point of view regarding future desirable trends are reported.This work was supported in part by the European Commission’s Horizon 2020 Framework Programme with the project REMODEL under Grant 870133 and in part by the Spanish Government under Grant PID2020-114819GB-I00.Peer ReviewedPostprint (published version
Development of an active vision system for robot inspection of complex objects
Dissertação de mestrado integrado em Engenharia Mecânica (área de especialização em Sistemas Mecatrónicos)The dissertation presented here is in the scope of the IntVis4Insp project between University of Minho
and the company Neadvance. It focuses on the development of a 3D hand tracking system that must be
capable of extracting the hand position and orientation to prepare a manipulator for automatic inspection
of leather pieces.
This work starts with a literature review about the two main methods for collecting the necessary data to
perform 3D hand tracking. These divide into glove-based methods and vision-based methods. The first
ones work with some kind of support mounted on the hand that holds all the necessary sensors to
measure the desired parameters. While the second ones recur to one or more cameras to capture the
hands and through computer vision algorithms track their position and configuration. The selected
method for this work was the vision-based method Openpose. For each recorded image, this application
can locate 21 hand keypoints on each hand that together form a skeleton of the hands.
This application is used in the tracking system developed throughout this dissertation. Its information is
used in a more complete pipeline where the location of those hand keypoints is crucial to track the hands
in videos of the demonstrated movements. These videos were recorded with an RGB-D camera, the
Microsoft Kinect, which provides a depth value for every RGB pixel recorded. With the depth information
and the 2D location of the hand keypoints in the images, it was possible to obtain the 3D world coordinates
of these points considering the pinhole camera model.
To define the hand, position a point is selected among the 21 for each hand, but for the hand orientation,
it was necessary to develop an auxiliary method called “Iterative Pose Estimation Method” (ITP), which
estimates the complete 3D pose of the hands. This method recurs only to the 2D locations of every hand
keypoint, and the complete 3D world coordinates of the wrists to estimate the right 3D world coordinates
of all the remaining points on the hand. This solution solves the problems related to hand occlusions that
a prone to happen due to the use of only one camera to record the inspection videos. Once the world
location of all the points in the hands is accurately estimated, their orientation can be defined by selecting
three points forming a plane.A dissertação aqui apresentada insere-se no âmbito do projeto IntVis4Insp entre a Universidade do Minho
e a empresa Neadavance, e foca-se no desenvolvimento de um sistema para extração da posição e
orientação das mãos no espaço para posterior auxílio na manipulação automática de peças de couro,
com recurso a manipuladores robóticos.
O trabalho inicia-se com uma revisão literária sobre os dois principais métodos existentes para efetuar a
recolha de dados necessária à monitorização da posição e orientação das mãos ao longo do tempo.
Estes dividem-se em métodos baseados em luvas ou visão. No caso dos primeiros, estes recorrem
normalmente a algum tipo de suporte montado na mão (ex.: luva em tecido), onde estão instalados todos
os sensores necessários para a medição dos parâmetros desejados. Relativamente a sistemas de visão
estes recorrem a uma câmara ou conjunto delas para capturar as mãos e por via de algoritmos de visão
por computador determinam a sua posição e configuração. Foi selecionado para este trabalho um
algoritmo de visão por computador denominado por Openpose. Este é capaz de, em cada imagem
gravada e para cada mão, localizar 21 pontos pertencentes ao seu esqueleto.
Esta aplicação é inserida no sistema de monitorização desenvolvido, sendo utilizada a sua informação
numa arquitetura mais completa onde é efetuada a extração da localização dos pontos chave de cada
mão nos vídeos de demonstração dos movimentos de inspeção. A gravação destes vídeos é efetuada
com uma câmara RGB-D, a Microsoft Kinect, que fornece um valor de profundidade para cada pixel RGB
gravado. Com os dados de profundidade e a localização dos pontos chave nas imagens foi possível obter
as coordenadas 3D no mundo destes pontos considerando o modelo pinhole para a câmara. No caso da
posição da mão é selecionado um ponto de entre os 21 para a definir ao longo do tempo, no entanto,
para o cálculo da orientação foi desenvolvido um método auxiliar para estimação da pose tridimensional
da mão denominado por “Iterative Pose Estimation Method” (ITP). Este método recorre aos dados 2D
do Openpose e às coordenadas 3D do pulso de cada mão para efetuar a correta estimação das
coordenadas 3D dos restantes pontos da mão. Isto permite essencialmente resolver problemas com
oclusões da mão, muito frequentes com o uso de uma só câmara na gravação dos vídeos. Uma vez
estimada corretamente a posição 3D no mundo dos vários pontos da mão, a sua orientação pode ser
definida com recurso a quaisquer três pontos que definam um plano
On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation
Biological and robotic grasp and manipulation are undeniably similar at the
level of mechanical task performance. However, their underlying fundamental
biological vs. engineering mechanisms are, by definition, dramatically
different and can even be antithetical. Even our approach to each is
diametrically opposite: inductive science for the study of biological systems
vs. engineering synthesis for the design and construction of robotic systems.
The past 20 years have seen several conceptual advances in both fields and the
quest to unify them. Chief among them is the reluctant recognition that their
underlying fundamental mechanisms may actually share limited common ground,
while exhibiting many fundamental differences. This recognition is particularly
liberating because it allows us to resolve and move beyond multiple paradoxes
and contradictions that arose from the initial reasonable assumption of a large
common ground. Here, we begin by introducing the perspective of neuromechanics,
which emphasizes that real-world behavior emerges from the intimate
interactions among the physical structure of the system, the mechanical
requirements of a task, the feasible neural control actions to produce it, and
the ability of the neuromuscular system to adapt through interactions with the
environment. This allows us to articulate a succinct overview of a few salient
conceptual paradoxes and contradictions regarding under-determined vs.
over-determined mechanics, under- vs. over-actuated control, prescribed vs.
emergent function, learning vs. implementation vs. adaptation, prescriptive vs.
descriptive synergies, and optimal vs. habitual performance. We conclude by
presenting open questions and suggesting directions for future research. We
hope this frank assessment of the state-of-the-art will encourage and guide
these communities to continue to interact and make progress in these important
areas
Data-Driven Grasp Synthesis - A Survey
We review the work on data-driven grasp synthesis and the methodologies for
sampling and ranking candidate grasps. We divide the approaches into three
groups based on whether they synthesize grasps for known, familiar or unknown
objects. This structure allows us to identify common object representations and
perceptual processes that facilitate the employed data-driven grasp synthesis
technique. In the case of known objects, we concentrate on the approaches that
are based on object recognition and pose estimation. In the case of familiar
objects, the techniques use some form of a similarity matching to a set of
previously encountered objects. Finally for the approaches dealing with unknown
objects, the core part is the extraction of specific features that are
indicative of good grasps. Our survey provides an overview of the different
methodologies and discusses open problems in the area of robot grasping. We
also draw a parallel to the classical approaches that rely on analytic
formulations.Comment: 20 pages, 30 Figures, submitted to IEEE Transactions on Robotic
- …