8 research outputs found
Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
In this paper we introduce Co-Fusion, a dense SLAM system that takes a live
stream of RGB-D images as input and segments the scene into different objects
(using either motion or semantic cues) while simultaneously tracking and
reconstructing their 3D shape in real time. We use a multiple model fitting
approach where each object can move independently from the background and still
be effectively tracked and its shape fused over time using only the information
from pixels associated with that object label. Previous attempts to deal with
dynamic scenes have typically considered moving regions as outliers, and
consequently do not model their shape or track their motion over time. In
contrast, we enable the robot to maintain 3D models for each of the segmented
objects and to improve them over time through fusion. As a result, our system
can enable a robot to maintain a scene description at the object level which
has the potential to allow interactions with its working environment; even in
the case of dynamic scenes.Comment: International Conference on Robotics and Automation (ICRA) 2017,
http://visual.cs.ucl.ac.uk/pubs/cofusion,
https://github.com/martinruenz/co-fusio
Object and Relation Centric Representations for Push Effect Prediction
Pushing is an essential non-prehensile manipulation skill used for tasks
ranging from pre-grasp manipulation to scene rearrangement, reasoning about
object relations in the scene, and thus pushing actions have been widely
studied in robotics. The effective use of pushing actions often requires an
understanding of the dynamics of the manipulated objects and adaptation to the
discrepancies between prediction and reality. For this reason, effect
prediction and parameter estimation with pushing actions have been heavily
investigated in the literature. However, current approaches are limited because
they either model systems with a fixed number of objects or use image-based
representations whose outputs are not very interpretable and quickly accumulate
errors. In this paper, we propose a graph neural network based framework for
effect prediction and parameter estimation of pushing actions by modeling
object relations based on contacts or articulations. Our framework is validated
both in real and simulated environments containing different shaped multi-part
objects connected via different types of joints and objects with different
masses. Our approach enables the robot to predict and adapt the effect of a
pushing action as it observes the scene. Further, we demonstrate 6D effect
prediction in the lever-up action in the context of robot-based hard-disk
disassembly.Comment: Project Page: https://fzaero.github.io/push_learning
Capturing Hand-Object Interaction and Reconstruction of Manipulated Objects
Hand motion capture with an RGB-D sensor gained recently a lot of research attention, however, even most recent approaches focus on the case of a single isolated hand. We focus instead on hands that interact with other hands or with a rigid or articulated object. Our framework successfully captures motion in such scenarios by combining a generative model with discriminatively trained salient points, collision detection and physics simulation to achieve a low tracking error with physically plausible poses. All components are unified in a single objective function that can be optimized with standard optimization techniques. We initially assume a-priori knowledge of the object’s shape and skeleton. In case of unknown object shape there are existing 3d reconstruction methods that capitalize on distinctive geometric or texture features. These methods though fail for textureless and highly symmetric objects like household articles, mechanical parts or toys. We show that extracting 3d hand motion for in-hand scanning e↵ectively facilitates the reconstruction of such objects and we fuse the rich additional information of hands into a 3d reconstruction pipeline. Finally, although shape reconstruction is enough for rigid objects, there is a lack of tools that build rigged models of articulated objects that deform realistically using RGB-D data. We propose a method that creates a fully rigged model consisting of a watertight mesh, embedded skeleton and skinning weights by employing a combination of deformable mesh tracking, motion segmentation based on spectral clustering and skeletonization based on mean curvature flow
An Integrated Approach to Visual Perception of Articulated Objects -- Data
These are the rosbag files that accompany the ICRA2016 Paper "An Integrated Approach to Visual Perception of Articulated Objects". The rosbags contain RGBD recordings of moving articualted objects such as a drawer or a globe. Additionally there is ground truth information about the shapes of the objects supplied.DFG, 260200664, Die Physical Exploration Challenge: Roboter, die lernen, Freiheitsgrade der Welt zu entdecken, zu bewegen und zu explorierenEC/H2020/645599/EU/Soft-bodied intelligence for Manipulation/SOM