1 research outputs found
Learning to Transfer Visual Effects from Videos to Images
We study the problem of animating images by transferring spatio-temporal
visual effects (such as melting) from a collection of videos. We tackle two
primary challenges in visual effect transfer: 1) how to capture the effect we
wish to distill; and 2) how to ensure that only the effect, rather than content
or artistic style, is transferred from the source videos to the input image. To
address the first challenge, we evaluate five loss functions; the most
promising one encourages the generated animations to have similar optical flow
and texture motions as the source videos. To address the second challenge, we
only allow our model to move existing image pixels from the previous frame,
rather than predicting unconstrained pixel values. This forces any visual
effects to occur using the input image's pixels, preventing unwanted artistic
style or content from the source video from appearing in the output. We
evaluate our method in objective and subjective settings, and show interesting
qualitative results which demonstrate objects undergoing atypical
transformations, such as making a face melt or a deer bloom