CAP-VSTNet: Content Affinity Preserved Versatile Style Transfer

Content affinity loss including feature and pixel affinity is a main problem which leads to artifacts in photorealistic and video style transfer. This paper proposes a new framework named CAP-VSTNet, which consists of a new reversible residual network and an unbiased linear transform module, for versatile style transfer. This reversible residual network can not only preserve content affinity but not introduce redundant information as traditional reversible networks, and hence facilitate better stylization. Empowered by Matting Laplacian training loss which can address the pixel affinity loss problem led by the linear transform, the proposed framework is applicable and effective on versatile style transfer. Extensive experiments show that CAP-VSTNet can produce better qualitative and quantitative results in comparison with the state-of-the-art methods.Comment: CVPR 202

Outdoor Dynamic 3-D Scene Reconstruction

Existing systems for 3D reconstruction from multiple view video use controlled indoor environments with uniform illumination and backgrounds to allow accurate segmentation of dynamic foreground objects. In this paper we present a portable system for 3D reconstruction of dynamic outdoor scenes which require relatively large capture volumes with complex backgrounds and non-uniform illumination. This is motivated by the demand for 3D reconstruction of natural outdoor scenes to support film and broadcast production. Limitations of existing multiple view 3D reconstruction techniques for use in outdoor scenes are identified. Outdoor 3D scene reconstruction is performed in three stages: (1) 3D background scene modelling using spherical stereo image capture; (2) multiple view segmentation of dynamic foreground objects by simultaneous video matting across multiple views; and (3) robust 3D foreground reconstruction and multiple view segmentation refinement in the presence of segmentation and calibration errors. Evaluation is performed on several outdoor productions with complex dynamic scenes including people and animals. Results demonstrate that the proposed approach overcomes limitations of previous indoor multiple view reconstruction approaches enabling high-quality free-viewpoint rendering and 3D reference models for production

Bayesian Optimization for Image Segmentation, Texture Flow Estimation and Image Deblurring

Ph.DDOCTOR OF PHILOSOPH

Disparity refinement process based on RANSAC plane fitting for machine vision applications

This paper presents a new disparity map refinement process for stereo matching algorithm and the refinement stage that will be implemented by partitioning the place or mask image and re-projected to the preliminary disparity images. This process is to refine the noise and sparse of initial disparity map from weakly textured. The plane fitting algorithm is using Random Sample Consensus. Two well-known stereo matching algorithms have been tested on this framework with different filtering techniques applied at disparity refinement stage. The framework is evaluated on three Middlebury datasets. The experimental results show that the proposed framework produces better-quality and more accurate than normal flow state-of-the-art stereo matching algorithms. The performance evaluations are based on standard image quality metrics i.e. structural similarity index measure, peak signal-to-noise ratio and mean square error.Keywords: computer vision; disparity refinement; image segmentation; RANSAC; stereo.

MoSculp: Interactive Visualization of Shape and Time

We present a system that allows users to visualize complex human motion via 3D motion sculptures---a representation that conveys the 3D structure swept by a human body as it moves through space. Given an input video, our system computes the motion sculptures and provides a user interface for rendering it in different styles, including the options to insert the sculpture back into the original video, render it in a synthetic scene or physically print it. To provide this end-to-end workflow, we introduce an algorithm that estimates that human's 3D geometry over time from a set of 2D images and develop a 3D-aware image-based rendering approach that embeds the sculpture back into the scene. By automating the process, our system takes motion sculpture creation out of the realm of professional artists, and makes it applicable to a wide range of existing video material. By providing viewers with 3D information, motion sculptures reveal space-time motion information that is difficult to perceive with the naked eye, and allow viewers to interpret how different parts of the object interact over time. We validate the effectiveness of this approach with user studies, finding that our motion sculpture visualizations are significantly more informative about motion than existing stroboscopic and space-time visualization methods.Comment: UIST 2018. Project page: http://mosculp.csail.mit.edu

Stereo matching on objects with fractional boundary.

Xiong, Wei.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references (leaves 56-61).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 2 --- Background Study --- p.6Chapter 2.1 --- Stereo matching --- p.6Chapter 2.2 --- Digital image matting --- p.8Chapter 2.3 --- Expectation Maximization --- p.9Chapter 3 --- Model Definition --- p.12Chapter 4 --- Initialization --- p.20Chapter 4.1 --- Initializing disparity --- p.20Chapter 4.2 --- Initializing alpha matte --- p.24Chapter 5 --- Optimization --- p.26Chapter 5.1 --- Expectation Step --- p.27Chapter 5.1.1 --- "Computing E((Pp(df = d1̐ưجθ(n),U))" --- p.28Chapter 5.1.2 --- "Computing E((Pp(db = d2̐ưجθ(n),U))" --- p.29Chapter 5.2 --- Maximization Step --- p.31Chapter 5.2.1 --- "Optimize α, given {F, B} fixed" --- p.34Chapter 5.2.2 --- "Optimize {F, B}, given α fixed" --- p.37Chapter 5.3 --- Computing Final Disparities --- p.40Chapter 6 --- Experiment Results --- p.42Chapter 7 --- Conclusion --- p.54Bibliography --- p.5