39,068 research outputs found
Deep Homography Estimation for Dynamic Scenes
Homography estimation is an important step in many computer vision problems.
Recently, deep neural network methods have shown to be favorable for this
problem when compared to traditional methods. However, these new methods do not
consider dynamic content in input images. They train neural networks with only
image pairs that can be perfectly aligned using homographies. This paper
investigates and discusses how to design and train a deep neural network that
handles dynamic scenes. We first collect a large video dataset with dynamic
content. We then develop a multi-scale neural network and show that when
properly trained using our new dataset, this neural network can already handle
dynamic scenes to some extent. To estimate a homography of a dynamic scene in a
more principled way, we need to identify the dynamic content. Since dynamic
content detection and homography estimation are two tightly coupled tasks, we
follow the multi-task learning principles and augment our multi-scale network
such that it jointly estimates the dynamics masks and homographies. Our
experiments show that our method can robustly estimate homography for
challenging scenarios with dynamic scenes, blur artifacts, or lack of textures.Comment: CVPR 2020, https://github.com/lcmhoang/hmg-dynamic
Object-Oriented Dynamics Learning through Multi-Level Abstraction
Object-based approaches for learning action-conditioned dynamics has
demonstrated promise for generalization and interpretability. However, existing
approaches suffer from structural limitations and optimization difficulties for
common environments with multiple dynamic objects. In this paper, we present a
novel self-supervised learning framework, called Multi-level Abstraction
Object-oriented Predictor (MAOP), which employs a three-level learning
architecture that enables efficient object-based dynamics learning from raw
visual observations. We also design a spatial-temporal relational reasoning
mechanism for MAOP to support instance-level dynamics learning and handle
partial observability. Our results show that MAOP significantly outperforms
previous methods in terms of sample efficiency and generalization over novel
environments for learning environment models. We also demonstrate that learned
dynamics models enable efficient planning in unseen environments, comparable to
true environment models. In addition, MAOP learns semantically and visually
interpretable disentangled representations.Comment: Accepted to the Thirthy-Fourth AAAI Conference On Artificial
Intelligence (AAAI), 202
- …