954 research outputs found
Video Inpainting by Jointly Learning Temporal Structure and Spatial Details
We present a new data-driven video inpainting method for recovering missing
regions of video frames. A novel deep learning architecture is proposed which
contains two sub-networks: a temporal structure inference network and a spatial
detail recovering network. The temporal structure inference network is built
upon a 3D fully convolutional architecture: it only learns to complete a
low-resolution video volume given the expensive computational cost of 3D
convolution. The low resolution result provides temporal guidance to the
spatial detail recovering network, which performs image-based inpainting with a
2D fully convolutional network to produce recovered video frames in their
original resolution. Such two-step network design ensures both the spatial
quality of each frame and the temporal coherence across frames. Our method
jointly trains both sub-networks in an end-to-end manner. We provide
qualitative and quantitative evaluation on three datasets, demonstrating that
our method outperforms previous learning-based video inpainting methods.Comment: Accepted by AAAI 201
An Internal Learning Approach to Video Inpainting
We propose a novel video inpainting algorithm that simultaneously
hallucinates missing appearance and motion (optical flow) information, building
upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network
architectures to enforce plausible texture in static images. In extending DIP
to video we make two important contributions. First, we show that coherent
video inpainting is possible without a priori training. We take a generative
approach to inpainting based on internal (within-video) learning without
reliance upon an external corpus of visual data to train a one-size-fits-all
model for the large space of general videos. Second, we show that such a
framework can jointly generate both appearance and flow, whilst exploiting
these complementary modalities to ensure mutual consistency. We show that
leveraging appearance statistics specific to each video achieves visually
plausible results whilst handling the challenging problem of long-term
consistency.Comment: Accepted by ICCV 2019. Website:
https://cs.stanford.edu/~haotianz/publications/video_inpainting
- …