39 research outputs found
Understanding of Object Manipulation Actions Using Human Multi-Modal Sensory Data
Object manipulation actions represent an important share of the Activities of
Daily Living (ADLs). In this work, we study how to enable service robots to use
human multi-modal data to understand object manipulation actions, and how they
can recognize such actions when humans perform them during human-robot
collaboration tasks. The multi-modal data in this study consists of videos,
hand motion data, applied forces as represented by the pressure patterns on the
hand, and measurements of the bending of the fingers, collected as human
subjects performed manipulation actions. We investigate two different
approaches. In the first one, we show that multi-modal signal (motion, finger
bending and hand pressure) generated by the action can be decomposed into a set
of primitives that can be seen as its building blocks. These primitives are
used to define 24 multi-modal primitive features. The primitive features can in
turn be used as an abstract representation of the multi-modal signal and
employed for action recognition. In the latter approach, the visual features
are extracted from the data using a pre-trained image classification deep
convolutional neural network. The visual features are subsequently used to
train the classifier. We also investigate whether adding data from other
modalities produces a statistically significant improvement in the classifier
performance. We show that both approaches produce a comparable performance.
This implies that image-based methods can successfully recognize human actions
during human-robot collaboration. On the other hand, in order to provide
training data for the robot so it can learn how to perform object manipulation
actions, multi-modal data provides a better alternative
Recognizing Intent in Collaborative Manipulation
Collaborative manipulation is inherently multimodal, with haptic
communication playing a central role. When performed by humans, it involves
back-and-forth force exchanges between the participants through which they
resolve possible conflicts and determine their roles. Much of the existing work
on collaborative human-robot manipulation assumes that the robot follows the
human. But for a robot to match the performance of a human partner it needs to
be able to take initiative and lead when appropriate. To achieve such
human-like performance, the robot needs to have the ability to (1) determine
the intent of the human, (2) clearly express its own intent, and (3) choose its
actions so that the dyad reaches consensus. This work proposes a framework for
recognizing human intent in collaborative manipulation tasks using force
exchanges. Grounded in a dataset collected during a human study, we introduce a
set of features that can be computed from the measured signals and report the
results of a classifier trained on our collected human-human interaction data.
Two metrics are used to evaluate the intent recognizer: overall accuracy and
the ability to correctly identify transitions. The proposed recognizer shows
robustness against the variations in the partner's actions and the confounding
effects due to the variability in grasp forces and dynamic effects of walking.
The results demonstrate that the proposed recognizer is well-suited for
implementation in a physical interaction control scheme
Motion Planning in Humans and Robots
We present a general framework for generating trajectories and actuator forces that will take a robot system from an initial configuration to a goal configuration in the presence of obstacles observed with noisy sensors. The central idea is to find the motion plan that optimizes a performance criterion dictated by specific task requirements. The approach is motivated by studies of human voluntary manipulation tasks that suggest that human motions can be described as solutions of certain optimization problems
A Geometric Approach to the Study of the Cartesian Stiffness Matrix
The stiffness of a rigid body subject to conservative forces and moments is described by a tensor, whose components are best described by a 6×6 Cartesian stiffness matrix. We derive an expression that is independent of the parameterization of the motion of the rigid body using methods of differential geometry. The components of the tensor with respect to a basis of twists are given by evaluating the tensor on a pair of basis twists. We show that this tensor depends on the choice of an affine connection on the Lie group, SE(3). In addition, we show that the definition of the Cartesian stiffness matrix used in the literature [2,6] implicitly assumes an asymmetric connection and this results in an asymmetric stiffness matrix in a general loaded configuration. We prove that by choosing a symmetric connection we always obtain a symmetric Cartesian stiffness matrix. Finally, we derive stiffness matrices for different connections and illustrate the calculations using numerical examples
Robots Taking Initiative in Collaborative Object Manipulation: Lessons from Physical Human-Human Interaction
Physical Human-Human Interaction (pHHI) involves the use of multiple sensory
modalities. Studies of communication through spoken utterances and gestures are
well established. Nevertheless, communication through force signals is not well
understood. In this paper, we focus on investigating the mechanisms employed by
humans during the negotiation through force signals, which is an integral part
of successful collaboration. Our objective is to use the insights to inform the
design of controllers for robot assistants. Specifically, we want to enable
robots to take the lead in collaboration. To achieve this goal, we conducted a
study to observe how humans behave during collaborative manipulation tasks.
During our preliminary data analysis, we discovered several new features that
help us better understand how the interaction progresses. From these features,
we identified distinct patterns in the data that indicate when a participant is
expressing their intent. Our study provides valuable insight into how humans
collaborate physically, which can help us design robots that behave more like
humans in such scenarios
Continuous Methods for Motion Planning
Motion planning for a mechanical system addresses the problem of finding a trajectory and actuator forces that are consistent with a given set of constraints and perform a desired task. In general, the problem is under-determined and admits a large number of solutions. The main claim of this dissertation is that a natural way to resolve the indeterminacy is to define performance of a motion and find a solution with the best performance. The motion planning problem is thus formulated as a variational problem. The proposed approach is continuous in the sense that the motion planning problem is not discretized. A distinction is made between dynamic and kinematic motion planning. Dynamic motion planning provides the actuator forces as part of the motion plan and requires finding a motion that is consistent with the dynamic equations of the system, satisfies a given set of equality and inequality constraints, and minimizes a chosen cost functional. In kinematic motion planning, dynamic equations of the system are not taken into account and it is therefore simpler. For dynamic motion planning, a novel numerical method for solving a variational problem is developed. The method combines discretization of the continuous problem motivated by the finite-element analysis with techniques from nonlinear programming. It is used to find smooth trajectories and actuator forces for two planar cooperating manipulators holding an object. The method is then extended for systems that change the dynamic equations as they move. An example of a simple grasping task illustrates that for such systems variational approach unifies motion planning and task planning. Kinematic motion planning is formulated as a variational problem on a Riemannian manifold. A Riemannian metric and an affine connection are introduced to define cost functionals that measure smoothness of trajectories. Kinematic motion planning is normally used in the task space, which can be represented by the group of spatial rigid body displacements, SE(3). It is shown how the group structure of SE(3) can be used to find smooth trajectories that have certain invariance properties with respect to the choice of the inertial and body fixed frames
Review of the Literature on Time-Optimal Control of Robotic Manipulators
A task that robotic manipulators most frequently perform is motion between specified points in the working space. It is therefore important that these motions are efficient. The presence of the obstacles and other requirements of the task often require that the path is specified in advance. Robot actuators cannot generate unlimited forces/torques so it is reasonable to ask how to traverse the prespecified path in minimum time so that the limits on the actuator torques are not violated. It can be shown that the motion which requires least time to traverse a path requires at least one actuator to operate on the boundary (maximum or minimum). Furthermore, if the path is parameterized, the equations describing the robot dynamics can be rewritten as functions of the path parameter and its first and second derivatives. In general, the actuator bounds will be transformed into the bounds on the acceleration along the path. These bounds will be functions of the velocity and position. It is pos..
Review of the Literature on Time-Optimal Control of Robotic Manipulators
A task that robotic manipulators most frequently perform is motion between specified points in the working space. It is therefore important that these motions are efficient. The presence of the obstacles and other requirements of the task often require that the path is specified in advance. Robot actuators cannot generate unlimited forces/torques so it is reasonable to ask how to traverse the prespecified path in minimum time so that the limits on the actuator torques are not violated. It can be shown that the motion which requires least time to traverse a path requires at least one actuator to operate on the boundary (maximum or minimum). Furthermore, if the path is parameterized, the equations describing the robot dynamics can be rewritten as functions of the path parameter and its first and second derivatives. In general, the actuator bounds will be transformed into the bounds on the acceleration along the path. These bounds will be functions of the velocity and position. It is poss..