Applying scenarios in user-centred design to develop a sketching interface for human modelling and animation

This paper presents our user and usability studies for applying scenarios in user-centred design to develop a sketching interface for virtual human modelling and animation. In this approach, we utilise the User Centred System Design (UCSD) strategy and spiral lifecycles to ensure system usability and functionalities. A series of usability techniques were employed. After the initial conceptual design, a preliminary user study (including questionnaires and sketching observations) was undertaken to establish the formal interface design. Second, an informal user test was conducted on the first prototype: a “sketch-based 3D stick figure animation interface”. Finally, a formal user evaluation (including performance tests, sketching observations, and interviews) was carried out on the latest version: a “sketch-based virtual human builder”. During this iterative process, various paper-based and electronic-based sketching scenarios were created, which were acted-out by users to help designers evoke and verify design ideas, identify users’ needs, and test the prototype interfaces in real contexts. Benefiting from applying the UCSD strategy and scenario-based design to develop a natural and supportive sketching interface, our investigation can be a useful instantiation for the design of other sketching interfaces where these techniques have not been widely acknowledged and utilised in the past

A sketch-based gesture interface for rough 3D stick figure animation

This paper introduces a novel gesture interface for sketching out rough 3D stick figure animation. This interface can allow users to draw stick figures with the system automatic assistance in figure proportion control. Given a 2D hand-drawn stick figure under a parallel view, there is a challenge to reconstruct a unique 3D pose from a set of candidates. Our system utilizes figure perspective rendering, and introduces the concept of ‘thickness contrast’ as a sketch gesture combined with some other constraints/assumptions for pose recovery. The resulting pose can be further corrected, based on physical constraints of human body. Once obtaining a series of 3D stick figure poses, user can easily sketch out motion paths and timing, and add their preferable sound/background. The resulting 3D animation can be automatically synthesized in VRML. This system has been tested on a variety of input devices: electric whiteboard, tablet PC, as well as a standard mouse

A sketching interface for 3D modeling of polyhedron

We present an intuitive and interactive freehand sketching interface for 3D polyhedrons reconstruction. The interface mimics sketching with pencil on paper and takes freehand sketches as input directly. The sketching environment is natural by allowing sketching with discontinuous, overlapping and multiple strokes. The input sketch is a natural line drawing with hidden lines removed that depicts a 3D object in an isometric view. The line drawing is interpreted by a series of 2D tidy-up processes to produce a vertex-edge graph for 3D reconstruction. A novel reconstruction approach based on three-line-junction analysis and planarity constraint is then used to approximate the 3D geometry and topology of the graph. The reconstructed object can be transformed so that it can be viewed from different viewpoints for interactive design or as immediate feedback to the designers. A new sketch can then be added to the existing 3D object, and reconstructed into 3D by referring to the existing 3D object from the current viewpoint. The incremental modeling enables a 3D object to be reconstructed from multiple sketching sessions from different viewpoints. However, the interface is limited to reconstructing trihedrons from sketches without T-junctions to avoid ambiguity in the hidden topology determination

Sketch-based virtual human modelling and animation

Animated virtual humans created by skilled artists play a remarkable role in today’s public entertainment. However, ordinary users are still treated as audiences due to the lack of appropriate expertise, equipment, and computer skills. We developed a new method and a novel sketching interface, which enable anyone who can draw to “sketch-out” 3D virtual humans and animation. We devised a “Stick FigureFleshing-outSkin Mapping” graphical pipeline, which decomposes the complexity of figure drawing and considerably boosts the modelling and animation efficiency. We developed a gesture-based method for 3D pose reconstruction from 2D stick figure drawings. We investigated a “Creative Model-based Method”, which performs a human perception process to transfer users’ 2D freehand sketches into 3D human bodies of various body sizes, shapes and fat distributions. Our current system supports character animation in various forms including articulated figure animation, 3D mesh model animation, and 2D contour/NPR animation with personalised drawing styles. Moreover, this interface also supports sketch-based crowd animation and 2D storyboarding of 3D multiple character interactions. A preliminary user study was conducted to support the overall system design. Our system has been formally tested by various users on Tablet PC. After minimal training, even a beginner can create vivid virtual humans and animate them within minutes

Sketching-out virtual humans: From 2d storyboarding to immediate 3d character animation

Virtual beings are playing a remarkable role in today’s public entertainment, while ordinary users are still treated as audiences due to the lack of appropriate expertise, equipment, and computer skills. In this paper, we present a fast and intuitive storyboarding interface, which enables users to sketch-out 3D virtual humans, 2D/3D animations, and character intercommunication. We devised an intuitive “stick figurefleshing-outskin mapping” graphical animation pipeline, which realises the whole process of key framing, 3D pose reconstruction, virtual human modelling, motion path/timing control, and the final animation synthesis by almost pure 2D sketching. A “creative model-based method” is developed, which emulates a human perception process, to generate the 3D human bodies of variational sizes, shapes, and fat distributions. Meanwhile, our current system also supports the sketch-based crowd animation and the storyboarding of the 3D multiple character intercommunication. This system has been formally tested by various users on Tablet PC. After minimal training, even a beginner can create vivid virtual humans and animate them within minutes

Investigating user preferences in utilizing a 2D paper or 3D sketch based interface for creating 3D virtual models

Computer modelling of 2D drawings is becoming increasingly popular in modern design as can be witnessed in the shift of modern computer modelling applications from software requiring specialised training to ones targeted for the general consumer market. Despite this, traditional sketching is still prevalent in design, particularly so in the early design stages. Thus, research trends in computer-aided modelling focus on the the development of sketch based interfaces that are as natural as possible. In this report, we present a hybrid sketch based interface which allows the user to make draw sketches using offline as well as online sketching modalities, displaying the 3D models in an immersive setup, thus linking the object interaction possible through immersive modelling to the flexibility allowed by paper-based sketching. The interface was evaluated in a user study which shows that such a hybrid system can be considered as having pragmatic and hedonic value.peer-reviewe

Human motion retrieval based on freehand sketch

In this paper, we present an integrated framework of human motion retrieval based on freehand sketch. With some simple rules, the user can acquire a desired motion by sketching several key postures. To retrieve efficiently and accurately by sketch, the 3D postures are projected onto several 2D planes. The limb direction feature is proposed to represent the input sketch and the projected-postures. Furthermore, a novel index structure based on k-d tree is constructed to index the motions in the database, which speeds up the retrieval process. With our posture-by-posture retrieval algorithm, a continuous motion can be got directly or generated by using a pre-computed graph structure. What's more, our system provides an intuitive user interface. The experimental results demonstrate the effectiveness of our method. © 2014 John Wiley & Sons, Ltd

Semi-Automated SVG Programming via Direct Manipulation

Direct manipulation interfaces provide intuitive and interactive features to a broad range of users, but they often exhibit two limitations: the built-in features cannot possibly cover all use cases, and the internal representation of the content is not readily exposed. We believe that if direct manipulation interfaces were to (a) use general-purpose programs as the representation format, and (b) expose those programs to the user, then experts could customize these systems in powerful new ways and non-experts could enjoy some of the benefits of programmable systems. In recent work, we presented a prototype SVG editor called Sketch-n-Sketch that offered a step towards this vision. In that system, the user wrote a program in a general-purpose lambda-calculus to generate a graphic design and could then directly manipulate the output to indirectly change design parameters (i.e. constant literals) in the program in real-time during the manipulation. Unfortunately, the burden of programming the desired relationships rested entirely on the user. In this paper, we design and implement new features for Sketch-n-Sketch that assist in the programming process itself. Like typical direct manipulation systems, our extended Sketch-n-Sketch now provides GUI-based tools for drawing shapes, relating shapes to each other, and grouping shapes together. Unlike typical systems, however, each tool carries out the user's intention by transforming their general-purpose program. This novel, semi-automated programming workflow allows the user to rapidly create high-level, reusable abstractions in the program while at the same time retaining direct manipulation capabilities. In future work, our approach may be extended with more graphic design features or realized for other application domains.Comment: In 29th ACM User Interface Software and Technology Symposium (UIST 2016