1 research outputs found
Customizing First Person Image Through Desired Actions
This paper studies a problem of inverse visual path planning: creating a
visual scene from a first person action. Our conjecture is that the spatial
arrangement of a first person visual scene is deployed to afford an action, and
therefore, the action can be inversely used to synthesize a new scene such that
the action is feasible. As a proof-of-concept, we focus on linking visual
experiences induced by walking.
A key innovation of this paper is a concept of ActionTunnel---a 3D virtual
tunnel along the future trajectory encoding what the wearer will visually
experience as moving into the scene. This connects two distinctive first person
images through similar walking paths. Our method takes a first person image
with a user defined future trajectory and outputs a new image that can afford
the future motion. The image is created by combining present and future
ActionTunnels in 3D where the missing pixels in adjoining area are computed by
a generative adversarial network. Our work can provide a travel across
different first person experiences in diverse real world scenes