4 research outputs found
Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches
Sketch-based image editing aims to synthesize and modify photos based on the
structural information provided by the human-drawn sketches. Since sketches are
difficult to collect, previous methods mainly use edge maps instead of sketches
to train models (referred to as edge-based models). However, sketches display
great structural discrepancy with edge maps, thus failing edge-based models.
Moreover, sketches often demonstrate huge variety among different users,
demanding even higher generalizability and robustness for the editing model to
work. In this paper, we propose Deep Plastic Surgery, a novel, robust and
controllable image editing framework that allows users to interactively edit
images using hand-drawn sketch inputs. We present a sketch refinement strategy,
as inspired by the coarse-to-fine drawing process of the artists, which we show
can help our model well adapt to casual and varied sketches without the need
for real sketch training data. Our model further provides a refinement level
control parameter that enables users to flexibly define how "reliable" the
input sketch should be considered for the final output, balancing between
sketch faithfulness and output verisimilitude (as the two goals might
contradict if the input sketch is drawn poorly). To achieve the multi-level
refinement, we introduce a style-based module for level conditioning, which
allows adaptive feature representations for different levels in a singe
network. Extensive experimental results demonstrate the superiority of our
approach in improving the visual quality and user controllablity of image
editing over the state-of-the-art methods
Faithful completion of images of scenic landmarks using internet images
Abstract—Previous works on image completion typically aim to produce visually plausible results rather than factually correct ones. In this paper, we propose an approach to faithfully complete the missing regions of an image. We assume that the input image is taken at a well-known landmark, so similar images taken at the same location can be easily found on the Internet. We first download thousands of images from the Internet using a text label provided by the user. Next, we apply two-step filtering to reduce them to a small set of candidate images for use as source images for completion. For each candidate image, a co-matching algorithm is used to find correspondences of both points and lines between the candidate image and the input image. These are used to find an optimal warp relating the two images. A completion result is obtained by blending the warped candidate image into the missing region of the input image. The completion results are ranked according to combination score, which considers both warping and blending energy, and the highest ranked ones are shown to the user. Experiments and results demonstrate that our method can faithfully complete images
180-degree Outpainting from a Single Image
Presenting context images to a viewer's peripheral vision is one of the most
effective techniques to enhance immersive visual experiences. However, most
images only present a narrow view, since the field-of-view (FoV) of standard
cameras is small. To overcome this limitation, we propose a deep learning
approach that learns to predict a 180{\deg} panoramic image from a narrow-view
image. Specifically, we design a foveated framework that applies different
strategies on near-periphery and mid-periphery regions. Two networks are
trained separately, and then are employed jointly to sequentially perform
narrow-to-90{\deg} generation and 90{\deg}-to-180{\deg} generation. The
generated outputs are then fused with their aligned inputs to produce expanded
equirectangular images for viewing. Our experimental results show that
single-view-to-panoramic image generation using deep learning is both feasible
and promising