DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling

Face modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this paper, we propose a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our network fuses both CNN and shape based features of the input sketch, and has two independent branches of fully connected layers generating independent subsets of coefficients for a bilinear face representation. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques.Comment: 12 pages, 16 figures, to appear in SIGGRAPH 201

Freeform User Interfaces for Graphical Computing

報告番号: 甲15222 ; 学位授与年月日: 2000-03-29 ; 学位の種別: 課程博士 ; 学位の種類: 博士(工学) ; 学位記番号: 博工第4717号 ; 研究科・専攻: 工学系研究科情報工学専

Natural Language Interfaces for Procedural Content Generation in Games

Mixed-Initiative Procedural Content Generation (MI-PCG) focuses on developing systems that allow users with diverse technical backgrounds to co-create interesting and novel game content in collaboration with a computational agent. These systems provide a front-end for users to interact with a generator by means of placing different constraints, or modifying a variety of the generator's parameters. While these systems provide significantly enhanced design support over traditional design tools, there exists areas of opportunity to address shortcomings in these systems such as high user interface complexity (too many controls presented, little feedback provided) and the lack of a model of designer intent (the system can reason over constraints but does not understand the expressive intent of the user). We believe that natural language interfaces can provide a way of addressing these areas by utilizing the expressiveness of natural language as an input for mixed-initiative systems in a way that it can reduce interface complexity by converting natural language queries into design space movements or constraints for the generator to act upon. By reducing all input to a single query the natural language interface can make the appropriate selection of parameters and controls that can result in the desired result for the user compared to the traditional modification of one control at a time in a traditional graphical user interface. Furthermore, the issue of designer intent can be addressed by creating a mapping of natural language concepts into a series of parameter combinations that allows for multi-dimensional movements in the design space of the generator, rather than manipulating a series of controls sequentially to achieve the same effect. In this thesis we explore the design and implementations of natural languages in MI-PCG systems through the development of a design methodology for encoding natural language understanding into MI-PCG systems and the implementation of two proof of concept systems named CADI and WATER4-NL for different use case scenarios such as automated game design and shader manipulation respectively. Furthermore a user study based evaluation of WATER4-NL and its results are presented and discussed

Advanced Focused Ion Beam: Preparation Optimization and Damage Mitigation

Focused Ion Beam (FIB) is an important analytical and sample modification technique in the field of electron and ion microscopy. It has been widely used in different kinds of applications including semiconductor device failure analysis, material science research, nanoscale 3D tomography, as well as microstructure prototyping and surface modification. Recent developments from the rapid growing industry and our frontier research have posted new challenges on the FIB technology itself. Higher resolution has been realized by state-of-the-art hardware infrastructures and less sample destruction has been achieved by efficient operation recipes. In this doctoral thesis, a study of advanced Focused Ion Beam sample preparation is presented, with the goal to prepare samples with low or no damage. The study is divided into two aspects according to various aspects in the FIB applications: sample damage and in-situ preparation. In the first aspect, sample damage, namely amorphization, ion implantation and FIB milling rate are investigated on crystalline silicon specimens with a gallium FIB tool. To study the ion-beam induced amorphous layer thickness under certain conditions, silicon specimens were prepared by FIB into specific geometry, so that the induced amorphous layer can be imaged and the thickness can be determined quantitatively using Transmission Electron Microscopy (TEM). Atom Probe Tomography (APT) was carried out to study the implanted ion concentration of gallium FIB prepared silicon specimens. In addition, the gallium FIB milling rate was also studied for a silicon substrate using Scanning Electron Microscopy (SEM). These experimental results provide detailed information of beam-sample interactions from the FIB sample preparation. In order to gain a systematic understanding of the processes, as well as to be able to predict the outcome of a specific FIB recipe, a physics model and an adapted algorithm (TRIDYN) based on Binary Collision Approximation (BCA) were used for the simulation of FIB processes. The predicted results based on simulations were compared with experiments. The proposed model was successfully validated by the experimental results, i.e., the TRIDYN algorithm has the capability to provide predictions for the multi- step FIB sample preparation process and the respective recipes. The other aspect involves a novel design of a hardware configuration of a SEM/FIB system add-on to perform in-situ surface modification tasks such as argon ion polishing of specimens. This Beam Induced Polishing System (BIPS) overcomes the disadvantages that some of the ex-situ methods have, and it completes some of the advanced FIB recipes for extremely thin and pristine specimens. In the thesis, the functionality of a BIPS system is explained in detail, and first experimental results are shown to demonstrate the proof of concept of the system. To summarize, this doctoral thesis presents an adapted algorithm, which is validated by experiments, to simulate the multi-step Focused Ion Beam process for recipes of low- damage sample preparation; A novel in-situ experiment system BIPS is also introduced, providing an option to complement SEM/FIB systems for advanced FIB sample preparation recipes

Interactive Evolutionary Algorithms for Image Enhancement and Creation

Image enhancement and creation, particularly for aesthetic purposes, are tasks for which the use of interactive evolutionary algorithms would seem to be well suited. Previous work has concentrated on the development of various aspects of the interactive evolutionary algorithms and their application to various image enhancement and creation problems. Robust evaluation of algorithmic design options in interactive evolutionary algorithms and the comparison of interactive evolutionary algorithms to alternative approaches to achieving the same goals is generally less well addressed. The work presented in this thesis is primarily concerned with different interactive evolutionary algorithms, search spaces, and operators for setting the input values required by image processing and image creation tasks. A secondary concern is determining when the use of the interactive evolutionary algorithm approach to image enhancement problems is warranted and how it compares with alternative approaches. Various interactive evolutionary algorithms were implemented and compared in a number of specifically devised experiments using tasks of varying complexity. A novel aspect of this thesis, with regards to other work in the study of interactive evolutionary algorithms, was that statistical analysis of the data gathered from the experiments was performed. This analysis demonstrated, contrary to popular assumption, that the choice of algorithm parameters, operators, search spaces, and even the underlying evolutionary algorithm has little effect on the quality of the resulting images or the time it takes to develop them. It was found that the interaction methods chosen when implementing the user interface of the interactive evolutionary algorithms had a greater influence on the performances of the algorithms

Artistic Content Representation and Modelling based on Visual Style Features

This thesis aims to understand visual style in the context of computer science, using traditionally intangible artistic properties to enhance existing content manipulation algorithms and develop new content creation methods. The developed algorithms can be used to apply extracted properties to other drawings automatically; transfer a selected style; categorise images based upon perceived style; build 3D models using style features from concept artwork; and other style-based actions that change our perception of an object without changing our ability to recognise it. The research in this thesis aims to provide the style manipulation abilities that are missing from modern digital art creation pipelines