Understanding of Object Manipulation Actions Using Human Multi-Modal Sensory Data

Object manipulation actions represent an important share of the Activities of Daily Living (ADLs). In this work, we study how to enable service robots to use human multi-modal data to understand object manipulation actions, and how they can recognize such actions when humans perform them during human-robot collaboration tasks. The multi-modal data in this study consists of videos, hand motion data, applied forces as represented by the pressure patterns on the hand, and measurements of the bending of the fingers, collected as human subjects performed manipulation actions. We investigate two different approaches. In the first one, we show that multi-modal signal (motion, finger bending and hand pressure) generated by the action can be decomposed into a set of primitives that can be seen as its building blocks. These primitives are used to define 24 multi-modal primitive features. The primitive features can in turn be used as an abstract representation of the multi-modal signal and employed for action recognition. In the latter approach, the visual features are extracted from the data using a pre-trained image classification deep convolutional neural network. The visual features are subsequently used to train the classifier. We also investigate whether adding data from other modalities produces a statistically significant improvement in the classifier performance. We show that both approaches produce a comparable performance. This implies that image-based methods can successfully recognize human actions during human-robot collaboration. On the other hand, in order to provide training data for the robot so it can learn how to perform object manipulation actions, multi-modal data provides a better alternative

Recognizing Intent in Collaborative Manipulation

Collaborative manipulation is inherently multimodal, with haptic communication playing a central role. When performed by humans, it involves back-and-forth force exchanges between the participants through which they resolve possible conflicts and determine their roles. Much of the existing work on collaborative human-robot manipulation assumes that the robot follows the human. But for a robot to match the performance of a human partner it needs to be able to take initiative and lead when appropriate. To achieve such human-like performance, the robot needs to have the ability to (1) determine the intent of the human, (2) clearly express its own intent, and (3) choose its actions so that the dyad reaches consensus. This work proposes a framework for recognizing human intent in collaborative manipulation tasks using force exchanges. Grounded in a dataset collected during a human study, we introduce a set of features that can be computed from the measured signals and report the results of a classifier trained on our collected human-human interaction data. Two metrics are used to evaluate the intent recognizer: overall accuracy and the ability to correctly identify transitions. The proposed recognizer shows robustness against the variations in the partner's actions and the confounding effects due to the variability in grasp forces and dynamic effects of walking. The results demonstrate that the proposed recognizer is well-suited for implementation in a physical interaction control scheme

Motion Planning in Humans and Robots

We present a general framework for generating trajectories and actuator forces that will take a robot system from an initial configuration to a goal configuration in the presence of obstacles observed with noisy sensors. The central idea is to find the motion plan that optimizes a performance criterion dictated by specific task requirements. The approach is motivated by studies of human voluntary manipulation tasks that suggest that human motions can be described as solutions of certain optimization problems

A Geometric Approach to the Study of the Cartesian Stiffness Matrix

The stiffness of a rigid body subject to conservative forces and moments is described by a tensor, whose components are best described by a 6×6 Cartesian stiffness matrix. We derive an expression that is independent of the parameterization of the motion of the rigid body using methods of differential geometry. The components of the tensor with respect to a basis of twists are given by evaluating the tensor on a pair of basis twists. We show that this tensor depends on the choice of an affine connection on the Lie group, SE(3). In addition, we show that the definition of the Cartesian stiffness matrix used in the literature [2,6] implicitly assumes an asymmetric connection and this results in an asymmetric stiffness matrix in a general loaded configuration. We prove that by choosing a symmetric connection we always obtain a symmetric Cartesian stiffness matrix. Finally, we derive stiffness matrices for different connections and illustrate the calculations using numerical examples

Robots Taking Initiative in Collaborative Object Manipulation: Lessons from Physical Human-Human Interaction

Physical Human-Human Interaction (pHHI) involves the use of multiple sensory modalities. Studies of communication through spoken utterances and gestures are well established. Nevertheless, communication through force signals is not well understood. In this paper, we focus on investigating the mechanisms employed by humans during the negotiation through force signals, which is an integral part of successful collaboration. Our objective is to use the insights to inform the design of controllers for robot assistants. Specifically, we want to enable robots to take the lead in collaboration. To achieve this goal, we conducted a study to observe how humans behave during collaborative manipulation tasks. During our preliminary data analysis, we discovered several new features that help us better understand how the interaction progresses. From these features, we identified distinct patterns in the data that indicate when a participant is expressing their intent. Our study provides valuable insight into how humans collaborate physically, which can help us design robots that behave more like humans in such scenarios

Equivalence of switching linear systems by bisimulation

A general notion of hybrid bisimulation is proposed for the class of switching linear systems. Connections between the notions of bisimulation-based equivalence, state-space equivalence, algebraic and input–output equivalence are investigated. An algebraic characterization of hybrid bisimulation and an algorithmic procedure converging in a finite number of steps to the maximal hybrid bisimulation are derived. Hybrid state space reduction is performed by hybrid bisimulation between the hybrid system and itself. By specializing the results obtained on bisimulation, also characterizations of simulation and abstraction are derived. Connections between observability, bisimulation-based reduction and simulation-based abstraction are studied.\ud \u

Invariant higher-order variational problems II

Motivated by applications in computational anatomy, we consider a second-order problem in the calculus of variations on object manifolds that are acted upon by Lie groups of smooth invertible transformations. This problem leads to solution curves known as Riemannian cubics on object manifolds that are endowed with normal metrics. The prime examples of such object manifolds are the symmetric spaces. We characterize the class of cubics on object manifolds that can be lifted horizontally to cubics on the group of transformations. Conversely, we show that certain types of non-horizontal geodesics on the group of transformations project to cubics. Finally, we apply second-order Lagrange--Poincar\'e reduction to the problem of Riemannian cubics on the group of transformations. This leads to a reduced form of the equations that reveals the obstruction for the projection of a cubic on a transformation group to again be a cubic on its object manifold.Comment: 40 pages, 1 figure. First version -- comments welcome

Feedback stability for dissipative switched systems

© 2017 The Authors. Published by Elsevier in co-operation with IFAC. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1016/j.ifacol.2017.08.843A method is proposed to infer Lyapunov and asymptotic stability properties for switching systems, under arbitrary continuous-state feedback. Continuous-time systems which are dissipative in the multiple-storage function sense are considered. A partition of the state space, induced by the cross-supply rates and the feedback function, is used to derive conditions for stability. It is argued that the conditions proposed here are more straightforward to check, when compared to those proposed by other approaches in the literature. Some numerical examples are offered to illustrate this point.Published versio

Inclusive and differential cross-section measurements of t\bartZ production in pp collisions at √s=13 TeV with the ATLAS detector, including EFT and spin-correlation interpretations

Measurements of both the inclusive and differential production cross sections of a top-quark-top-antiquark pair in association with a Z boson (tt¯Z) are presented. Final states with two, three or four isolated leptons (electrons or muons) are targeted. The measurements use the data recorded by the ATLAS detector in pp collisions at s√=13 TeV at the Large Hadron Collider during the years 2015-2018, corresponding to an integrated luminosity of 140 fb−1. The inclusive cross section is measured to be σtt¯Z=0.86±0.04 (stat.)±0.04 (syst.) pb and found to be in agreement with the most advanced Standard Model predictions. The differential measurements are presented as a function of a number of observables that probe the kinematics of the tt¯Z system. Both the absolute and normalised differential cross-section measurements are performed at particle level and parton level for specific fiducial volumes, and are compared with NLO+NNLL theoretical predictions. The results are interpreted in the framework of Standard Model effective field theory and used to set limits on a large number of dimension-6 operators involving the top quark. The first measurement of spin correlations in tt¯Z events is presented: the results are in agreement with the Standard Model expectations, and the null hypothesis of no spin correlations is disfavoured with a significance of 1.8 standard deviations

Observation of quantum entanglement in top-quark pairs using the ATLAS detector

We report the highest-energy observation of entanglement, in top−antitop quark events produced at the Large Hadron Collider, using a proton−proton collision data set with a center-of-mass energy of s√=13 TeV and an integrated luminosity of 140 fb−1 recorded with the ATLAS experiment. Spin entanglement is detected from the measurement of a single observable D, inferred from the angle between the charged leptons in their parent top- and antitop-quark rest frames. The observable is measured in a narrow interval around the top−antitop quark production threshold, where the entanglement detection is expected to be significant. It is reported in a fiducial phase space defined with stable particles to minimize the uncertainties that stem from limitations of the Monte Carlo event generators and the parton shower model in modelling top-quark pair production. The entanglement marker is measured to be D=−0.547±0.002 (stat.)±0.021 (syst.) for 340<mtt¯<380 GeV. The observed result is more than five standard deviations from a scenario without entanglement and hence constitutes both the first observation of entanglement in a pair of quarks and the highest-energy observation of entanglement to date