Efficient sketch-based creation of detailed character models through data-driven mesh deformations

Creation of detailed character models is a very challenging task in animation production. Sketch-based character model creation from a 3D template provides a promising solution. However, how to quickly find correct correspondences between user's drawn sketches and the 3D template model, how to efficiently deform the 3D template model to exactly match user's drawn sketches, and realize real-time interactive modeling is still an open topic. In this paper, we propose a new approach and develop a user interface to effectively tackle this problem. Our proposed approach includes using user's drawn sketches to retrieve a most similar 3D template model from our dataset and marrying human's perception and interactions with computer's highly efficient computing to extract occluding and silhouette contours of the 3D template model and find correct correspondences quickly. We then combine skeleton-based deformation and mesh editing to deform the 3D template model to fit user's drawn sketches and create new and detailed 3D character models. The results presented in this paper demonstrate the effectiveness and advantages of our proposed approach and usefulness of our developed user interface

Wave propagation and imaging in structured optical media

Structured optical media, usually characterized by periodic patterns of inhomogeneities in bulk materials, provide a new approach to ultimate control of wave propagation with possible practical applications: from distributed feedback lasers by diffraction gratings, to highly nonlinear performance for super-continuum generation, to fiber-optic telecommunications by microstructured photonic crystal fibers, to invisibility cloaking, to super-resolution imaging with metamaterials etc. In particular, structured optical media allow to manipulate the wave propagation and dispersion. In this thesis, we focus on engineering the propagation phase dispersion by modulating the compositions and dimensions of the periodic elements. By tailoring the dispersion in momentum space, we can obtain new optical properties of the structured media and show applications in optical imaging. In this work, we present a novel zeroth-order transmission resonance in hyperbolic medium with a Fabry-Perot geometry, which allows to control the propagation phase by subwavelength elements. This approach can also be extended to periodic structures and be applied to improve the performance of imaging systems. In particular, we show a negative refraction lens based on the photonic hypercrystals, which possesses a nearly constant negative refractive index and can be used to substantially reduce the image aberrations. We apply similar ideas in purely dielectric photonic crystals and demonstrate the phenomenon of conical refraction, which allows a new approach to imaging. In particular, it offers a new method of optical phase retrieval that enables a single simultaneous measurement and guarantees a rapid recovery to the true solution

Deep Learning on Lie Groups for Skeleton-based Action Recognition

In recent years, skeleton-based action recognition has become a popular 3D classification problem. State-of-the-art methods typically first represent each motion sequence as a high-dimensional trajectory on a Lie group with an additional dynamic time warping, and then shallowly learn favorable Lie group features. In this paper we incorporate the Lie group structure into a deep network architecture to learn more appropriate Lie group features for 3D action recognition. Within the network structure, we design rotation mapping layers to transform the input Lie group features into desirable ones, which are aligned better in the temporal domain. To reduce the high feature dimensionality, the architecture is equipped with rotation pooling layers for the elements on the Lie group. Furthermore, we propose a logarithm mapping layer to map the resulting manifold data into a tangent space that facilitates the application of regular output layers for the final classification. Evaluations of the proposed network for standard 3D human action recognition datasets clearly demonstrate its superiority over existing shallow Lie group feature learning methods as well as most conventional deep learning methods.Comment: Accepted to CVPR 201

Dance-the-music : an educational platform for the modeling, recognition and audiovisual monitoring of dance steps using spatiotemporal motion templates

In this article, a computational platform is presented, entitled “Dance-the-Music”, that can be used in a dance educational context to explore and learn the basics of dance steps. By introducing a method based on spatiotemporal motion templates, the platform facilitates to train basic step models from sequentially repeated dance figures performed by a dance teacher. Movements are captured with an optical motion capture system. The teachers’ models can be visualized from a first-person perspective to instruct students how to perform the specific dance steps in the correct manner. Moreover, recognition algorithms-based on a template matching method can determine the quality of a student’s performance in real time by means of multimodal monitoring techniques. The results of an evaluation study suggest that the Dance-the-Music is effective in helping dance students to master the basics of dance figures