2 research outputs found
Semantic View Synthesis
We tackle a new problem of semantic view synthesis -- generating
free-viewpoint rendering of a synthesized scene using a semantic label map as
input. We build upon recent advances in semantic image synthesis and view
synthesis for handling photographic image content generation and view
extrapolation. Direct application of existing image/view synthesis methods,
however, results in severe ghosting/blurry artifacts. To address the drawbacks,
we propose a two-step approach. First, we focus on synthesizing the color and
depth of the visible surface of the 3D scene. We then use the synthesized color
and depth to impose explicit constraints on the multiple-plane image (MPI)
representation prediction process. Our method produces sharp contents at the
original view and geometrically consistent renderings across novel viewpoints.
The experiments on numerous indoor and outdoor images show favorable results
against several strong baselines and validate the effectiveness of our
approach.Comment: ECCV 2020. Project: https://hhsinping.github.io/svs/index.html Colab:
https://colab.research.google.com/drive/1iT5PfK7zl1quAOwC227GfBjieFMVHjI
Modeling Artistic Workflows for Image Generation and Editing
People often create art by following an artistic workflow involving multiple
stages that inform the overall design. If an artist wishes to modify an earlier
decision, significant work may be required to propagate this new decision
forward to the final artwork. Motivated by the above observations, we propose a
generative model that follows a given artistic workflow, enabling both
multi-stage image generation as well as multi-stage image editing of an
existing piece of art. Furthermore, for the editing scenario, we introduce an
optimization process along with learning-based regularization to ensure the
edited image produced by the model closely aligns with the originally provided
image. Qualitative and quantitative results on three different artistic
datasets demonstrate the effectiveness of the proposed framework on both image
generation and editing tasks.Comment: ECCV 2020. Code: https://github.com/hytseng0509/ArtEditin