MonoPerfCap: Human Performance Capture from Monocular Video

We present the first marker-less approach for temporally coherent 3D performance capture of a human with general clothing from monocular video. Our approach reconstructs articulated human skeleton motion as well as medium-scale non-rigid surface deformations in general scenes. Human performance capture is a challenging problem due to the large range of articulation, potentially fast motion, and considerable non-rigid deformations, even from multi-view data. Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem. We tackle these challenges by a novel approach that employs sparse 2D and 3D human pose detections from a convolutional neural network using a batch-based pose estimation strategy. Joint recovery of per-batch motion allows to resolve the ambiguities of the monocular reconstruction problem based on a low dimensional trajectory subspace. In addition, we propose refinement of the surface geometry based on fully automatically extracted silhouettes to enable medium-scale non-rigid alignment. We demonstrate state-of-the-art performance capture results that enable exciting applications such as video editing and free viewpoint video, previously infeasible from monocular video. Our qualitative and quantitative evaluation demonstrates that our approach significantly outperforms previous monocular methods in terms of accuracy, robustness and scene complexity that can be handled.Comment: Accepted to ACM TOG 2018, to be presented on SIGGRAPH 201

Single-picture reconstruction and rendering of trees for plausible vegetation synthesis

State-of-the-art approaches for tree reconstruction either put limiting constraints on the input side (requiring multiple photographs, a scanned point cloud or intensive user input) or provide a representation only suitable for front views of the tree. In this paper we present a complete pipeline for synthesizing and rendering detailed trees from a single photograph with minimal user effort. Since the overall shape and appearance of each tree is recovered from a single photograph of the tree crown, artists can benefit from georeferenced images to populate landscapes with native tree species. A key element of our approach is a compact representation of dense tree crowns through a radial distance map. Our first contribution is an automatic algorithm for generating such representations from a single exemplar image of a tree. We create a rough estimate of the crown shape by solving a thin-plate energy minimization problem, and then add detail through a simplified shape-from-shading approach. The use of seamless texture synthesis results in an image-based representation that can be rendered from arbitrary view directions at different levels of detail. Distant trees benefit from an output-sensitive algorithm inspired on relief mapping. For close-up trees we use a billboard cloud where leaflets are distributed inside the crown shape through a space colonization algorithm. In both cases our representation ensures efficient preservation of the crown shape. Major benefits of our approach include: it recovers the overall shape from a single tree image, involves no tree modeling knowledge and minimal authoring effort, and the associated image-based representation is easy to compress and thus suitable for network streaming.Peer ReviewedPostprint (author's final draft

3D scanning of cultural heritage with consumer depth cameras

Three dimensional reconstruction of cultural heritage objects is an expensive and time-consuming process. Recent consumer real-time depth acquisition devices, like Microsoft Kinect, allow very fast and simple acquisition of 3D views. However 3D scanning with such devices is a challenging task due to the limited accuracy and reliability of the acquired data. This paper introduces a 3D reconstruction pipeline suited to use consumer depth cameras as hand-held scanners for cultural heritage objects. Several new contributions have been made to achieve this result. They include an ad-hoc filtering scheme that exploits the model of the error on the acquired data and a novel algorithm for the extraction of salient points exploiting both depth and color data. Then the salient points are used within a modified version of the ICP algorithm that exploits both geometry and color distances to precisely align the views even when geometry information is not sufficient to constrain the registration. The proposed method, although applicable to generic scenes, has been tuned to the acquisition of sculptures and in this connection its performance is rather interesting as the experimental results indicate

Hybridization of silhouette rendering and pen-and-ink illustration of non-photorealistic rendering technique for 3D object

This study proposes a hybrid of Non-photorealistic Rendering techniques. Nonphotorealistic Rendering (NPR) covers one part in computer graphics that caters towards generating many kinds of 2D digital art style from 3D data, for instance output that looks like painting and drawing. NPR includes the painterly, interpretative, expressive and artistic styles, among others. NPR research deal with different issues such as the stylization that are driven by human perception, the science and art that were brought together and being harmonized with techniques used. Some of approaches used in NPR were discussed such as cartoon rendering, watercolour painting, silhouette rendering, penand- ink illustration and so on. A plan for hybridization of NPR techniques is proposed between silhouette rendering techniques and pen-and-ink illustration for this study. The integration process of these rendering techniques takes on the lighting mapping and also the construction of colour region of the model in order to ensure the pen-and-ink illustration texture can be implemented into the object. The evaluation process is based on the visualization of the image from the hybridization process. Based on findings, the hybridization of NPR technique was able to create interesting results and considered as an alternative in producing new variety of visualization image in NPR

Penumbra maps: approximate soft shadows in real-time

Journal ArticleGenerating soft shadows quickly is difficult. Few techniques have enough flexibility to interactively render soft shadows in scenes with arbitrarily complex occluders and receivers. This paper introduces the penumbra map, which extends current shadow map techniques to interactively approximate soft shadows. Using object silhouette edges, as seen from the center of an area light, a map is generated containing approximate penumbral regions. Rendering requires two lookups, one into each the penumbra and shadow maps. Penumbra maps allow arbitrary dynamic models to easily shadow themselves and other nearby complex objects with plausible penumbrae

Penumbra maps

technical reportGenerating soft shadows quickly is difficult. Few techniques have enough flexibility to interactively render soft shadows in scenes with arbitrarily complex occluders and receivers. This paper introduces the penumbra map, which extends current shadow map techniques to interactively approximate soft shadows. Using object silhouette edges, as seen from the center of an area light, a map is generated containing approximate penumbral regions. Rendering requires two lookups, one into each the penumbra and shadow maps. Penumbra maps allow arbitrary dynamic models to easily shadow themselves and other nearby complex objects with plausible penumbrae

Shadow Mapping or Shadow Volume?

In this paper two techniques of shadow generation are described. Volume shadow is geometric base but shadow mapping is image base. Silhouette detection is a most expensive step to create volume shadow. Two algorithms to recognize silhouette are introduced. Stencil buffer and Z- buffer are two other tools for creating shadow by volume shadow technique. Both algorithms are implemented in virtual environment with moveable light source. Triangular method and the Visible-non visible method are introduced. The recent traditional silhouette detection and implementation techniques used in volume shadow algorithm are improved. With introduce flowchart of both algorithms, the last volume shadow algorithm using stencil buffer is rewritten. A very simple algorithm to create volume shadow is proposed. The last shadow mapping algorithm is rewritten. These techniques are poised to bring realism into commercial games. It may be use in virtual reality applications

Interactive real-time three-dimensional visualisation of virtual textiles

Virtual textile databases provide a cost-efficient alternative to the use of existing hardcover sample catalogues. By taking advantage of the high performance features offered by the latest generation of programmable graphics accelerator boards, it is possible to combine photometric stereo methods with 3D visualisation methods to implement a virtual textile database. In this thesis, we investigate and combine rotation invariant texture retrieval with interactive visualisation techniques. We use a 3D surface representation that is a generic data representation that allows us to combine real-time interactive 3D visualisation methods with present day texture retrieval methods. We begin by investigating the most suitable data format for the 3D surface representation and identify relief-mapping combined with Bézier surfaces as the most suitable 3D surface representations for our needs, and go on to describe how these representation can be combined for real-time rendering. We then investigate ten different methods of implementing rotation invariant texture retrieval using feature vectors. These results show that first order statistics in the form of histogram data are very effective for discriminating colour albedo information, while rotation invariant gradient maps are effective for distinguishing between different types of micro-geometry using either first or second order statistics.Engineering and physical Sciences Research (EPSRC