150 research outputs found
The State of Lifelong Learning in Service Robots: Current Bottlenecks in Object Perception and Manipulation
Service robots are appearing more and more in our daily life. The development
of service robots combines multiple fields of research, from object perception
to object manipulation. The state-of-the-art continues to improve to make a
proper coupling between object perception and manipulation. This coupling is
necessary for service robots not only to perform various tasks in a reasonable
amount of time but also to continually adapt to new environments and safely
interact with non-expert human users. Nowadays, robots are able to recognize
various objects, and quickly plan a collision-free trajectory to grasp a target
object in predefined settings. Besides, in most of the cases, there is a
reliance on large amounts of training data. Therefore, the knowledge of such
robots is fixed after the training phase, and any changes in the environment
require complicated, time-consuming, and expensive robot re-programming by
human experts. Therefore, these approaches are still too rigid for real-life
applications in unstructured environments, where a significant portion of the
environment is unknown and cannot be directly sensed or controlled. In such
environments, no matter how extensive the training data used for batch
learning, a robot will always face new objects. Therefore, apart from batch
learning, the robot should be able to continually learn about new object
categories and grasp affordances from very few training examples on-site.
Moreover, apart from robot self-learning, non-expert users could interactively
guide the process of experience acquisition by teaching new concepts, or by
correcting insufficient or erroneous concepts. In this way, the robot will
constantly learn how to help humans in everyday tasks by gaining more and more
experiences without the need for re-programming
Discovering Affordances Through Perception and Manipulation
International audienceConsidering perception as an observation process only is the very reason for which robotic perception methods are to date unable to provide a general capacity of scene understanding. Related work in neuroscience has shown that there is a strong relationship between perception and action. We believe that considering perception in relation to action requires to interpret the scene in terms of the agent's own potential capabilities. In this paper, we propose a Bayesian approach for learning sensorimotor representations through the interaction between action and observation capabilities. We represent the notion of affordance as a probabilistic relation between three elements: objects, actions and effects. Experiments for affordances discovery were performed on a real robotic platform in an unsupervised way assuming a limited set of innate capabilities. Results show dependency relations that connect the three elements in a common frame: affordances. The increasing number of interactions and observations results in a Bayesian network that captures the relationships between them. The learned representation can be used for prediction tasks
Active Vision for Scene Understanding
Visual perception is one of the most important sources of information for both humans and robots. A particular challenge is the acquisition and interpretation of complex unstructured scenes. This work contributes to active vision for humanoid robots. A semantic model of the scene is created, which is extended by successively changing the robot\u27s view in order to explore interaction possibilities of the scene
Efficient Belief Propagation for Perception and Manipulation in Clutter
Autonomous service robots are required to perform tasks in common human indoor environments. To achieve goals associated with these tasks, the robot should continually perceive, reason its environment, and plan to manipulate objects, which we term as goal-directed manipulation. Perception remains the most challenging aspect of all stages, as common indoor environments typically pose problems in recognizing objects under inherent occlusions with physical interactions among themselves. Despite recent progress in the field of robot perception, accommodating perceptual uncertainty due to partial observations remains challenging and needs to be addressed to achieve the desired autonomy.
In this dissertation, we address the problem of perception under uncertainty for robot manipulation in cluttered environments using generative inference methods. Specifically, we aim to enable robots to perceive partially observable environments by maintaining an approximate probability distribution as a belief over possible scene hypotheses. This belief representation captures uncertainty resulting from inter-object occlusions and physical interactions, which are inherently present in clutterred indoor environments. The research efforts presented in this thesis are towards developing appropriate state representations and inference techniques to generate and maintain such belief over contextually plausible scene states. We focus on providing the following features to generative inference while addressing the challenges due to occlusions: 1) generating and maintaining plausible scene hypotheses, 2) reducing the inference search space that typically grows exponentially with respect to the number of objects in a scene, 3) preserving scene hypotheses over continual observations.
To generate and maintain plausible scene hypotheses, we propose physics informed scene estimation methods that combine a Newtonian physics engine within a particle based generative inference framework. The proposed variants of our method with and without a Monte Carlo step showed promising results on generating and maintaining plausible hypotheses under complete occlusions. We show that estimating such scenarios would not be possible by the commonly adopted 3D registration methods without the notion of a physical context that our method provides.
To scale up the context informed inference to accommodate a larger number of objects, we describe a factorization of scene state into object and object-parts to perform collaborative particle-based inference. This resulted in the Pull Message Passing for Nonparametric Belief Propagation (PMPNBP) algorithm that caters to the demands of the high-dimensional multimodal nature of cluttered scenes while being computationally tractable. We demonstrate that PMPNBP is orders of magnitude faster than the state-of-the-art Nonparametric Belief Propagation method. Additionally, we show that PMPNBP successfully estimates poses of articulated objects under various simulated occlusion scenarios.
To extend our PMPNBP algorithm for tracking object states over continuous observations, we explore ways to propose and preserve hypotheses effectively over time. This resulted in an augmentation-selection method, where hypotheses are drawn from various proposals followed by the selection of a subset using PMPNBP that explained the current state of the objects. We discuss and analyze our augmentation-selection method with its counterparts in belief propagation literature. Furthermore, we develop an inference pipeline for pose estimation and tracking of articulated objects in clutter. In this pipeline, the message passing module with the augmentation-selection method is informed by segmentation heatmaps from a trained neural network. In our experiments, we show that our proposed pipeline can effectively maintain belief and track articulated objects over a sequence of observations under occlusion.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163159/1/kdesingh_1.pd
Data-Driven Grasp Synthesis - A Survey
We review the work on data-driven grasp synthesis and the methodologies for
sampling and ranking candidate grasps. We divide the approaches into three
groups based on whether they synthesize grasps for known, familiar or unknown
objects. This structure allows us to identify common object representations and
perceptual processes that facilitate the employed data-driven grasp synthesis
technique. In the case of known objects, we concentrate on the approaches that
are based on object recognition and pose estimation. In the case of familiar
objects, the techniques use some form of a similarity matching to a set of
previously encountered objects. Finally for the approaches dealing with unknown
objects, the core part is the extraction of specific features that are
indicative of good grasps. Our survey provides an overview of the different
methodologies and discusses open problems in the area of robot grasping. We
also draw a parallel to the classical approaches that rely on analytic
formulations.Comment: 20 pages, 30 Figures, submitted to IEEE Transactions on Robotic
Active Vision for Scene Understanding
Visual perception is one of the most important sources of information for both humans and robots. A particular challenge is the acquisition and interpretation of complex unstructured scenes. This work contributes to active vision for humanoid robots. A semantic model of the scene is created, which is extended by successively changing the robot's view in order to explore interaction possibilities of the scene
Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policies
Grasping objects of different shapes and sizes - a foundational, effortless
skill for humans - remains a challenging task in robotics. Although model-based
approaches can predict stable grasp configurations for known object models,
they struggle to generalize to novel objects and often operate in a
non-interactive open-loop manner. In this work, we present a reinforcement
learning framework that learns the interactive grasping of various
geometrically distinct real-world objects by continuously controlling an
anthropomorphic robotic hand. We explore several explicit representations of
object geometry as input to the policy. Moreover, we propose to inform the
policy implicitly through signed distances and show that this is naturally
suited to guide the search through a shaped reward component. Finally, we
demonstrate that the proposed framework is able to learn even in more
challenging conditions, such as targeted grasping from a cluttered bin.
Necessary pre-grasping behaviors such as object reorientation and utilization
of environmental constraints emerge in this case. Videos of learned interactive
policies are available at https://maltemosbach.github.
io/geometry_aware_grasping_policies
Integrating Vision and Physical Interaction for Discovery, Segmentation and Grasping of Unknown Objects
In dieser Arbeit werden Verfahren der Bildverarbeitung und die Fähigkeit
humanoider Roboter, mit ihrer Umgebung physisch zu interagieren, in engem
Zusammenspiel eingesetzt, um unbekannte Objekte zu identifizieren, sie vom
Hintergrund und anderen Objekten zu trennen, und letztendlich zu greifen.
Im Verlauf dieser interaktiven Exploration werden auĂźerdem Eigenschaften
des Objektes wie etwa sein Aussehen und seine Form ermittelt
- …