267 research outputs found
The Meaning of Action:a review on action recognition and mapping
In this paper, we analyze the different approaches taken to date within the computer vision, robotics and artificial intelligence communities for the representation, recognition, synthesis and understanding of action. We deal with action at different levels of complexity and provide the reader with the necessary related literature references. We put the literature references further into context and outline a possible interpretation of action by taking into account the different aspects of action recognition, action synthesis and task-level planning
DAMM: Directionality-Aware Mixture Model Parallel Sampling for Efficient Dynamical System Learning
The Linear Parameter Varying Dynamical System (LPV-DS) is a promising
framework for learning stable time-invariant motion policies in robot control.
By employing statistical modeling and semi-definite optimization, LPV-DS
encodes complex motions via non-linear DS, ensuring the robustness and
stability of the system. However, the current LPV-DS scheme faces challenges in
accurately interpreting trajectory data while maintaining model efficiency and
computational efficiency. To address these limitations, we propose the
Directionality-aware Mixture Model (DAMM), a new statistical model that
leverages Riemannian metric on -dimensional sphere , and
efficiently incorporates non-Euclidean directional information with position.
Additionally, we introduce a hybrid Markov chain Monte Carlo method that
combines the Gibbs Sampling and the Split/Merge Proposal, facilitating parallel
computation and enabling faster inference for near real-time learning
performance. Through extensive empirical validation, we demonstrate that the
improved LPV-DS framework with DAMM is capable of producing
physically-meaningful representations of the trajectory data and improved
performance of the generated DS while showcasing significantly enhanced
learning speed compared to its previous iterations
Programming by Demonstration on Riemannian Manifolds
This thesis presents a Riemannian approach to Programming by Demonstration (PbD).
It generalizes an existing PbD method from Euclidean manifolds to Riemannian manifolds.
In this abstract, we review the objectives, methods and contributions of the presented
approach.
OBJECTIVES
PbD aims at providing a user-friendly method for skill transfer between human and
robot. It enables a user to teach a robot new tasks using few demonstrations. In order
to surpass simple record-and-replay, methods for PbD need to \u2018understand\u2019 what to
imitate; they need to extract the functional goals of a task from the demonstration data.
This is typically achieved through the application of statisticalmethods.
The variety of data encountered in robotics is large. Typical manipulation tasks involve
position, orientation, stiffness, force and torque data. These data are not solely
Euclidean. Instead, they originate from a variety of manifolds, curved spaces that are
only locally Euclidean. Elementary operations, such as summation, are not defined on
manifolds. Consequently, standard statistical methods are not well suited to analyze
demonstration data that originate fromnon-Euclidean manifolds. In order to effectively
extract what-to-imitate, methods for PbD should take into account the underlying geometry
of the demonstration manifold; they should be geometry-aware.
Successful task execution does not solely depend on the control of individual task
variables. By controlling variables individually, a task might fail when one is perturbed
and the others do not respond. Task execution also relies on couplings among task variables.
These couplings describe functional relations which are often called synergies. In
order to understand what-to-imitate, PbDmethods should be able to extract and encode
synergies; they should be synergetic.
In unstructured environments, it is unlikely that tasks are found in the same scenario
twice. The circumstances under which a task is executed\u2014the task context\u2014are more
likely to differ each time it is executed. Task context does not only vary during task execution,
it also varies while learning and recognizing tasks. To be effective, a robot should
be able to learn, recognize and synthesize skills in a variety of familiar and unfamiliar
contexts; this can be achieved when its skill representation is context-adaptive.
THE RIEMANNIAN APPROACH
In this thesis, we present a skill representation that is geometry-aware, synergetic and
context-adaptive. The presented method is probabilistic; it assumes that demonstrations
are samples from an unknown probability distribution. This distribution is approximated
using a Riemannian GaussianMixtureModel (GMM).
Instead of using the \u2018standard\u2019 Euclidean Gaussian, we rely on the Riemannian Gaussian\u2014
a distribution akin the Gaussian, but defined on a Riemannian manifold. A Riev
mannian manifold is a manifold\u2014a curved space which is locally Euclidean\u2014that provides
a notion of distance. This notion is essential for statistical methods as such methods
rely on a distance measure. Examples of Riemannian manifolds in robotics are: the
Euclidean spacewhich is used for spatial data, forces or torques; the spherical manifolds,
which can be used for orientation data defined as unit quaternions; and Symmetric Positive
Definite (SPD) manifolds, which can be used to represent stiffness and manipulability.
The Riemannian Gaussian is intrinsically geometry-aware. Its definition is based on
the geometry of the manifold, and therefore takes into account the manifold curvature.
In robotics, the manifold structure is often known beforehand. In the case of PbD, it follows
from the structure of the demonstration data. Like the Gaussian distribution, the
Riemannian Gaussian is defined by a mean and covariance. The covariance describes
the variance and correlation among the state variables. These can be interpreted as local
functional couplings among state variables: synergies. This makes the Riemannian
Gaussian synergetic. Furthermore, information encoded in multiple Riemannian Gaussians
can be fused using the Riemannian product of Gaussians. This feature allows us to
construct a probabilistic context-adaptive task representation.
CONTRIBUTIONS
In particular, this thesis presents a generalization of existing methods of PbD, namely
GMM-GMR and TP-GMM. This generalization involves the definition ofMaximum Likelihood
Estimate (MLE), Gaussian conditioning and Gaussian product for the Riemannian
Gaussian, and the definition of ExpectationMaximization (EM) and GaussianMixture
Regression (GMR) for the Riemannian GMM. In this generalization, we contributed
by proposing to use parallel transport for Gaussian conditioning. Furthermore, we presented
a unified approach to solve the aforementioned operations using aGauss-Newton
algorithm. We demonstrated how synergies, encoded in a Riemannian Gaussian, can be
transformed into synergetic control policies using standard methods for LinearQuadratic
Regulator (LQR). This is achieved by formulating the LQR problem in a (Euclidean) tangent
space of the Riemannian manifold. Finally, we demonstrated how the contextadaptive
Task-Parameterized Gaussian Mixture Model (TP-GMM) can be used for context
inference\u2014the ability to extract context from demonstration data of known tasks.
Our approach is the first attempt of context inference in the light of TP-GMM. Although
effective, we showed that it requires further improvements in terms of speed and reliability.
The efficacy of the Riemannian approach is demonstrated in a variety of scenarios.
In shared control, the Riemannian Gaussian is used to represent control intentions of a
human operator and an assistive system. Doing so, the properties of the Gaussian can
be employed to mix their control intentions. This yields shared-control systems that
continuously re-evaluate and assign control authority based on input confidence. The
context-adaptive TP-GMMis demonstrated in a Pick & Place task with changing pick and
place locations, a box-taping task with changing box sizes, and a trajectory tracking task
typically found in industr
Learning Generalizable Manipulation Policies with Object-Centric 3D Representations
We introduce GROOT, an imitation learning method for learning robust policies
with object-centric and 3D priors. GROOT builds policies that generalize beyond
their initial training conditions for vision-based manipulation. It constructs
object-centric 3D representations that are robust toward background changes and
camera views and reason over these representations using a transformer-based
policy. Furthermore, we introduce a segmentation correspondence model that
allows policies to generalize to new objects at test time. Through
comprehensive experiments, we validate the robustness of GROOT policies against
perceptual variations in simulated and real-world environments. GROOT's
performance excels in generalization over background changes, camera viewpoint
shifts, and the presence of new object instances, whereas both state-of-the-art
end-to-end learning methods and object proposal-based approaches fall short. We
also extensively evaluate GROOT policies on real robots, where we demonstrate
the efficacy under very wild changes in setup. More videos and model details
can be found in the appendix and the project website:
https://ut-austin-rpl.github.io/GROOT .Comment: Accepted at the 7th Annual Conference on Robot Learning (CoRL), 2023
in Atlanta, U
Robot Learning from Demonstration in Robotic Assembly: A Survey
Learning from demonstration (LfD) has been used to help robots to implement manipulation tasks autonomously, in particular, to learn manipulation behaviors from observing the motion executed by human demonstrators. This paper reviews recent research and development in the field of LfD. The main focus is placed on how to demonstrate the example behaviors to the robot in assembly operations, and how to extract the manipulation features for robot learning and generating imitative behaviors. Diverse metrics are analyzed to evaluate the performance of robot imitation learning. Specifically, the application of LfD in robotic assembly is a focal point in this paper
- …