156 research outputs found
Push to know! -- Visuo-Tactile based Active Object Parameter Inference with Dual Differentiable Filtering
For robotic systems to interact with objects in dynamic environments, it is
essential to perceive the physical properties of the objects such as shape,
friction coefficient, mass, center of mass, and inertia. This not only eases
selecting manipulation action but also ensures the task is performed as
desired. However, estimating the physical properties of especially novel
objects is a challenging problem, using either vision or tactile sensing. In
this work, we propose a novel framework to estimate key object parameters using
non-prehensile manipulation using vision and tactile sensing. Our proposed
active dual differentiable filtering (ADDF) approach as part of our framework
learns the object-robot interaction during non-prehensile object push to infer
the object's parameters. Our proposed method enables the robotic system to
employ vision and tactile information to interactively explore a novel object
via non-prehensile object push. The novel proposed N-step active formulation
within the differentiable filtering facilitates efficient learning of the
object-robot interaction model and during inference by selecting the next best
exploratory push actions (where to push? and how to push?). We extensively
evaluated our framework in simulation and real-robotic scenarios, yielding
superior performance to the state-of-the-art baseline.Comment: 8 pages. Accepted at IROS 202
Modeling motor control in continuous-time Active Inference: a survey
The way the brain selects and controls actions is still widely debated.
Mainstream approaches based on Optimal Control focus on stimulus-response
mappings that optimize cost functions. Ideomotor theory and cybernetics propose
a different perspective: they suggest that actions are selected and controlled
by activating action effects and by continuously matching internal predictions
with sensations. Active Inference offers a modern formulation of these ideas,
in terms of inferential mechanisms and prediction-error-based control, which
can be linked to neural mechanisms of living organisms. This article provides a
technical illustration of Active Inference models in continuous time and a
brief survey of Active Inference models that solve four kinds of control
problems; namely, the control of goal-directed reaching movements, active
sensing, the resolution of multisensory conflict during movement and the
integration of decision-making and motor control. Crucially, in Active
Inference, all these different facets of motor control emerge from the same
optimization process - namely, the minimization of Free Energy - and do not
require designing separate cost functions. Therefore, Active Inference provides
a unitary perspective on various aspects of motor control that can inform both
the study of biological control mechanisms and the design of artificial and
robotic systems
HARMONIOUS -- Human-like reactive motion control and multimodal perception for humanoid robots
For safe and effective operation of humanoid robots in human-populated
environments, the problem of commanding a large number of Degrees of Freedom
(DoF) while simultaneously considering dynamic obstacles and human proximity
has still not been solved. We present a new reactive motion controller that
commands two arms of a humanoid robot and three torso joints (17 DoF in total).
We formulate a quadratic program that seeks joint velocity commands respecting
multiple constraints while minimizing the magnitude of the velocities. We
introduce a new unified treatment of obstacles that dynamically maps visual and
proximity (pre-collision) and tactile (post-collision) obstacles as additional
constraints to the motion controller, in a distributed fashion over surface of
the upper-body of the iCub robot (with 2000 pressure-sensitive receptors). The
bio-inspired controller: (i) produces human-like minimum jerk movement
profiles; (ii) gives rise to a robot with whole-body visuo-tactile awareness,
resembling peripersonal space representations. The controller was extensively
experimentally validated, including a physical human-robot interaction
scenario.Comment: 14 pages, 7 figure
Of Priors and Particles: Structured and Distributed Approaches to Robot Perception and Control
Applications of robotic systems have expanded significantly in their scope, moving beyond the caged predictability of industrial automation and towards more open, unstructured environments. These agents must learn to reliably perceive their surroundings, efficiently integrate new information and quickly adapt to dynamic perturbations. To accomplish this, we require solutions which can effectively incorporate prior knowledge while maintaining the generality of learned representations. These systems must also contend with uncertainty in both their perception of the world and in predicting possible future outcomes. Efficient methods for probabilistic inference are then key to realizing robust, adaptive behavior.
This thesis will first examine data-driven approaches for learning and combining perceptual models for both visual and tactile sensor modalities, common in robotics. Modern variational inference methods will then be examined in the context of online optimization and stochastic optimal control. Specifically, this thesis will contribute (1) data-driven visual and tactile perceptual models leveraging kinematic and dynamic priors, (2) a framework for joint inference with visuo-tactile sensing, (3) a family of particle-based, variational model predictive control and planning algorithms, and (4) a distributed inference scheme for online model adaptation.Ph.D
Visuo-Haptic Grasping of Unknown Objects through Exploration and Learning on Humanoid Robots
Die vorliegende Arbeit befasst sich mit dem Greifen unbekannter Objekte durch humanoide Roboter. Dazu werden visuelle Informationen mit haptischer Exploration kombiniert, um Greifhypothesen zu erzeugen. Basierend auf simulierten Trainingsdaten wird außerdem eine Greifmetrik gelernt, welche die Erfolgswahrscheinlichkeit der Greifhypothesen bewertet und die mit der größten geschätzten Erfolgswahrscheinlichkeit auswählt. Diese wird verwendet, um Objekte mit Hilfe einer reaktiven Kontrollstrategie zu greifen. Die zwei Kernbeiträge der Arbeit sind zum einen die haptische Exploration von unbekannten Objekten und zum anderen das Greifen von unbekannten Objekten mit Hilfe einer neuartigen datengetriebenen Greifmetrik
On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation
Biological and robotic grasp and manipulation are undeniably similar at the
level of mechanical task performance. However, their underlying fundamental
biological vs. engineering mechanisms are, by definition, dramatically
different and can even be antithetical. Even our approach to each is
diametrically opposite: inductive science for the study of biological systems
vs. engineering synthesis for the design and construction of robotic systems.
The past 20 years have seen several conceptual advances in both fields and the
quest to unify them. Chief among them is the reluctant recognition that their
underlying fundamental mechanisms may actually share limited common ground,
while exhibiting many fundamental differences. This recognition is particularly
liberating because it allows us to resolve and move beyond multiple paradoxes
and contradictions that arose from the initial reasonable assumption of a large
common ground. Here, we begin by introducing the perspective of neuromechanics,
which emphasizes that real-world behavior emerges from the intimate
interactions among the physical structure of the system, the mechanical
requirements of a task, the feasible neural control actions to produce it, and
the ability of the neuromuscular system to adapt through interactions with the
environment. This allows us to articulate a succinct overview of a few salient
conceptual paradoxes and contradictions regarding under-determined vs.
over-determined mechanics, under- vs. over-actuated control, prescribed vs.
emergent function, learning vs. implementation vs. adaptation, prescriptive vs.
descriptive synergies, and optimal vs. habitual performance. We conclude by
presenting open questions and suggesting directions for future research. We
hope this frank assessment of the state-of-the-art will encourage and guide
these communities to continue to interact and make progress in these important
areas
- …