6,510 research outputs found
Metric Learning for Generalizing Spatial Relations to New Objects
Human-centered environments are rich with a wide variety of spatial relations
between everyday objects. For autonomous robots to operate effectively in such
environments, they should be able to reason about these relations and
generalize them to objects with different shapes and sizes. For example, having
learned to place a toy inside a basket, a robot should be able to generalize
this concept using a spoon and a cup. This requires a robot to have the
flexibility to learn arbitrary relations in a lifelong manner, making it
challenging for an expert to pre-program it with sufficient knowledge to do so
beforehand. In this paper, we address the problem of learning spatial relations
by introducing a novel method from the perspective of distance metric learning.
Our approach enables a robot to reason about the similarity between pairwise
spatial relations, thereby enabling it to use its previous knowledge when
presented with a new relation to imitate. We show how this makes it possible to
learn arbitrary spatial relations from non-expert users using a small number of
examples and in an interactive manner. Our extensive evaluation with real-world
data demonstrates the effectiveness of our method in reasoning about a
continuous spectrum of spatial relations and generalizing them to new objects.Comment: Accepted at the 2017 IEEE/RSJ International Conference on Intelligent
Robots and Systems. The new Freiburg Spatial Relations Dataset and a demo
video of our approach running on the PR-2 robot are available at our project
website: http://spatialrelations.cs.uni-freiburg.d
Optimization Beyond the Convolution: Generalizing Spatial Relations with End-to-End Metric Learning
To operate intelligently in domestic environments, robots require the ability
to understand arbitrary spatial relations between objects and to generalize
them to objects of varying sizes and shapes. In this work, we present a novel
end-to-end approach to generalize spatial relations based on distance metric
learning. We train a neural network to transform 3D point clouds of objects to
a metric space that captures the similarity of the depicted spatial relations,
using only geometric models of the objects. Our approach employs gradient-based
optimization to compute object poses in order to imitate an arbitrary target
relation by reducing the distance to it under the learned metric. Our results
based on simulated and real-world experiments show that the proposed method
enables robots to generalize spatial relations to unknown objects over a
continuous spectrum.Comment: Accepted for publication at ICRA2018. Supplementary Video:
http://spatialrelations.cs.uni-freiburg.de
SPIDA: Abstracting and generalizing layout design cases
Abstraction and generalization of layout design cases generate new knowledge that is more widely applicable to use than specific design cases. The abstraction and generalization of design cases into hierarchical levels of abstractions provide the designer with the flexibility to apply any level of abstract and generalized knowledge for a new layout design problem. Existing case-based layout learning (CBLL) systems abstract and generalize cases into single levels of abstractions, but not into a hierarchy. In this paper, we propose a new approach, termed customized viewpoint - spatial (CV-S), which supports the generalization and abstraction of spatial layouts into hierarchies along with a supporting system, SPIDA (SPatial Intelligent Design Assistant)
Geometric deep learning: going beyond Euclidean data
Many scientific fields study data with an underlying structure that is a
non-Euclidean space. Some examples include social networks in computational
social sciences, sensor networks in communications, functional networks in
brain imaging, regulatory networks in genetics, and meshed surfaces in computer
graphics. In many applications, such geometric data are large and complex (in
the case of social networks, on the scale of billions), and are natural targets
for machine learning techniques. In particular, we would like to use deep
neural networks, which have recently proven to be powerful tools for a broad
range of problems from computer vision, natural language processing, and audio
analysis. However, these tools have been most successful on data with an
underlying Euclidean or grid-like structure, and in cases where the invariances
of these structures are built into networks used to model them. Geometric deep
learning is an umbrella term for emerging techniques attempting to generalize
(structured) deep neural models to non-Euclidean domains such as graphs and
manifolds. The purpose of this paper is to overview different examples of
geometric deep learning problems and present available solutions, key
difficulties, applications, and future research directions in this nascent
field
Learning Object Placements For Relational Instructions by Hallucinating Scene Representations
Robots coexisting with humans in their environment and performing services
for them need the ability to interact with them. One particular requirement for
such robots is that they are able to understand spatial relations and can place
objects in accordance with the spatial relations expressed by their user. In
this work, we present a convolutional neural network for estimating pixelwise
object placement probabilities for a set of spatial relations from a single
input image. During training, our network receives the learning signal by
classifying hallucinated high-level scene representations as an auxiliary task.
Unlike previous approaches, our method does not require ground truth data for
the pixelwise relational probabilities or 3D models of the objects, which
significantly expands the applicability in practical applications. Our results
obtained using real-world data and human-robot experiments demonstrate the
effectiveness of our method in reasoning about the best way to place objects to
reproduce a spatial relation. Videos of our experiments can be found at
https://youtu.be/zaZkHTWFMKMComment: Accepted at the 2020 IEEE International Conference on Robotics and
Automation (ICRA). Video at https://www.youtube.com/watch?v=zaZkHTWFMK
Detecting Human-Object Interactions via Functional Generalization
We present an approach for detecting human-object interactions (HOIs) in
images, based on the idea that humans interact with functionally similar
objects in a similar manner. The proposed model is simple and efficiently uses
the data, visual features of the human, relative spatial orientation of the
human and the object, and the knowledge that functionally similar objects take
part in similar interactions with humans. We provide extensive experimental
validation for our approach and demonstrate state-of-the-art results for HOI
detection. On the HICO-Det dataset our method achieves a gain of over 2.5%
absolute points in mean average precision (mAP) over state-of-the-art. We also
show that our approach leads to significant performance gains for zero-shot HOI
detection in the seen object setting. We further demonstrate that using a
generic object detector, our model can generalize to interactions involving
previously unseen objects.Comment: AAAI 202
Learning Action Maps of Large Environments via First-Person Vision
When people observe and interact with physical spaces, they are able to
associate functionality to regions in the environment. Our goal is to automate
dense functional understanding of large spaces by leveraging sparse activity
demonstrations recorded from an ego-centric viewpoint. The method we describe
enables functionality estimation in large scenes where people have behaved, as
well as novel scenes where no behaviors are observed. Our method learns and
predicts "Action Maps", which encode the ability for a user to perform
activities at various locations. With the usage of an egocentric camera to
observe human activities, our method scales with the size of the scene without
the need for mounting multiple static surveillance cameras and is well-suited
to the task of observing activities up-close. We demonstrate that by capturing
appearance-based attributes of the environment and associating these attributes
with activity demonstrations, our proposed mathematical framework allows for
the prediction of Action Maps in new environments. Additionally, we offer a
preliminary glance of the applicability of Action Maps by demonstrating a
proof-of-concept application in which they are used in concert with activity
detections to perform localization.Comment: To appear at CVPR 201
- …