DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling

Face modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this paper, we propose a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our network fuses both CNN and shape based features of the input sketch, and has two independent branches of fully connected layers generating independent subsets of coefficients for a bilinear face representation. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques.Comment: 12 pages, 16 figures, to appear in SIGGRAPH 201

Bare-handed 3D drawing in augmented reality

Head-mounted augmented reality (AR) enables embodied in situ drawing in three dimensions (3D).We explore 3D drawing interactions based on uninstrumented, unencumbered (bare) hands that preserve the user’s ability to freely navigate and interact with the physical environment. We derive three alternative interaction techniques supporting bare-handed drawing in AR from the literature and by analysing several envisaged use cases. The three interaction techniques are evaluated in a controlled user study examining three distinct drawing tasks: planar drawing, path description, and 3D object reconstruction. The results indicate that continuous freehand drawing supports faster line creation than the control point-based alternatives, although with reduced accuracy. User preferences for the different techniques are mixed and vary considerably between the different tasks, highlighting the value of diverse and flexible interactions. The combined effectiveness of these three drawing techniques is illustrated in an example application of 3D AR drawing

Design Ideation: The Conceptual Sketch in a Digital Design Culture

Design ideation, or the generating and developing of design ideas is central to designing in both education and practice. But what is the impact of digital technology on conceptual tools, notably the traditional pen and paper sketch? Initial research, in the Introduction highlights how freehand drawing is under pressure from digital technology both in design schools and industry yet educators and practitioners alike seem uncertain how best to deal with the digital challenge. The Literature review, in which the investigation is likened to a sprawling rhizome, covers both practical and philosophical concerns about conceptual tools thereby extending the notion of sketching beyond pen and paper, to what is called sketcherly ways of designing. To find out how to capture designers' use of conceptual tools, the spectrum of Design methodology was explored in which the case study method combined with protocol study emerged as the research strategy to be tested in a Pilot study. The pilot outcome set the scene for a protocol study with first-year design students which, through a series of Ideation workshops, illuminates how sketching together with verbalisation is a powerful combination for conceptualisation. In this, the workshop format emerged as an effective means for encouraging novice designers to develop ideation skills. The pilot was also instrumental for conducting a Multiple case study comprising five second year design students and five recent design graduates working in industry in the domains of fashion, architecture, graphics, product and general design. Using self-reporting and interviews the cases illuminate real uses of conceptual tools situated in everyday designing that reveal the multifaceted yet unpredictable character of ideation. The Summary and Conclusion discuss the future role of the conceptual sketch in digital design environments and suggest how scholarship in the context of teaching and learning design can bridge the worlds of design schools and professional practice. In this, computer-aided ideation, CAI, emerged as a promising research field

Enhancing Freehand Sketching in Industrial Design: Description and Implementation of a Drawing Methodology for More Effective Representations

Freehand sketching is an important part of the design process that allows one to communicate in a quick and gestural way the first ideas about new concepts and is a medium for graphic thinking. It is important for architects and designers because it is a mechanism of representation, conceptualization, and abstraction for the communication between the creators and their audience. All academic courses related to industrial design include subjects aimed at acquiring skills in the use of manual tools of graphic representation, recognizing their importance in the integral training of the designer. However, sometimes the methodologies implemented in some subjects fail to develop adequately the skills of the students, who finish their studies with shortcomings in the field of graphic representation. This paper describes exercises that are part of a methodology designed to help students of industrial design acquire the skills to make an agile and effective use of freehand sketching. Through different uses of the elements of formal expression, the exercises address topics such as shape analysis, composition, light, color, and descriptive illustration. The methodology is applied experimentally in a subject of the bachelor’s degree in industrial design and product development engineering at Universitat Jaume I, introducing the students to different instruments and techniques of sketching and proposing various enriching ways of direct observation of the objectual reality that surrounds them. The paper concludes by evaluating the positive impact of the implemented methodology

Investigating user preferences in utilizing a 2D paper or 3D sketch based interface for creating 3D virtual models

Computer modelling of 2D drawings is becoming increasingly popular in modern design as can be witnessed in the shift of modern computer modelling applications from software requiring specialised training to ones targeted for the general consumer market. Despite this, traditional sketching is still prevalent in design, particularly so in the early design stages. Thus, research trends in computer-aided modelling focus on the the development of sketch based interfaces that are as natural as possible. In this report, we present a hybrid sketch based interface which allows the user to make draw sketches using offline as well as online sketching modalities, displaying the 3D models in an immersive setup, thus linking the object interaction possible through immersive modelling to the flexibility allowed by paper-based sketching. The interface was evaluated in a user study which shows that such a hybrid system can be considered as having pragmatic and hedonic value.peer-reviewe

Improving Visualization Skills in Engineering Education

This article analyzes the importance of visualization skills in engineering education. It proposes a dual approach based on computer graphics applications using both Web-based graphic applications and a sketch based modeling system. It addresses the importance of spatial abilities in the context of engineering education and the available techniques for evaluating these abilities from a psychological point of view. It then reviews some Web resources conceived to help students improve their spatial abilities and presents two educational applications. Finally, it presents a pilot study carried out at La Laguna University

Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-made Objects

We present a novel freehand sketch beautification method, which takes as input a freely drawn sketch of a man-made object and automatically beautifies it both geometrically and structurally. Beautifying a sketch is challenging because of its highly abstract and heavily diverse drawing manner. Existing methods are usually confined to the distribution of their limited training samples and thus cannot beautify freely drawn sketches with rich variations. To address this challenge, we adopt a divide-and-combine strategy. Specifically, we first parse an input sketch into semantic components, beautify individual components by a learned part beautification module based on part-level implicit manifolds, and then reassemble the beautified components through a structure beautification module. With this strategy, our method can go beyond the training samples and handle novel freehand sketches. We demonstrate the effectiveness of our system with extensive experiments and a perceptive study.Comment: 13 figure