Markerless motion capture for 3D human model animation using depth camera

3D animation is created using keyframe based system in 3D animation software such as Blender and Maya. Due to the long time interval and the need of high expertise in 3D animation, motion capture devices were used as an alternative and Microsoft Kinect v2 sensor is one of them. This research analyses the capabilities of the Kinect sensor in producing 3D human model animations using motion capture and keyframe based animation system in reference to a live motion performance. The quality, time interval and cost of both animation results were compared. The experimental result shows that motion capture system with Kinect sensor consumed less time (only 2.6%) and cost (30%) in the long run (10 minutes of animation) compare to keyframe-based system, but it produced lower quality animation. This was due to the lack of body detection accuracy when there is obstruction. Moreover, the sensor’s constant assumption that the performer’s body faces forward made it unreliable to be used for a wide variety of movements. Furthermore, standard test defined in this research covers most body parts’ movements to evaluate other motion capture system

The Stretch-Engine: A Method for Creating Exaggeration in Animation Through Squash and Stretch

Animators exaggerate character motion to emphasize personality and actions. Exaggeration is expressed by pushing a character’s pose, changing the action’s timing, or by changing a character’s form. This last method, referred to as squash and stretch, creates the most noticeable change in exaggeration. However, without practice, squash and stretch can adversely affect the animation. This work introduces a method to create exaggeration in motion by focusing solely on squash and stretch to control changes in a character’s form. It does this by displaying a limbs' path of motion and altering the shape of that path to create a change in the limb’s form. This paper provides information on tools that exist to create animation and exaggeration, then discusses the functionality and effectiveness of these tools and how they influenced the design of the Stretch-Engine. The Stretch-Engine is a prototype tool developed to demonstrate this approach and is designed to be integrated into an existing animation software, Maya. The Stretch-Engine contains a bipedal-humanoid rig with controls necessary for animation and the ability to squash and stretch. It can be accessed through a user interface that allows the animator to control squash and stretch by changing the shape of generated paths of motion. This method is then evaluated by comparing animations of realistic motion to versions created with the Stretch-Engine. These stretched versions displayed exaggerated results for their realistic counterparts, creating similar effects to Looney Tunes animation. This method fits within the animator’s workflow and helps new artists visualize and control squash and stretch to create exaggeration

Evaluation of graphical user interfaces for augmented reality based manual assembly support

Augmented reality (AR) technology is advancing rapidly and promises benefits to a wide variety of applications&mdashincluding manual assembly and maintenance tasks. This thesis addresses the design of user interfaces for AR applications, focusing specifically on information presentation interface elements for assembly tasks. A framework was developed and utilized to understand and classify these elements, as well as to evaluate numerous existing AR assembly interfaces from literature. Furthermore, a user study was conducted to investigate the strengths and weaknesses of concrete and abstract AR interface elements in an assembly scenario, as well as to compare AR assembly instructions against common paper-based assembly instructions. The results of this study supported, at least partially, the three hypotheses that concrete AR elements are more suitable to convey part manipulation information than abstract AR elements, that concrete AR and paper-based instructions lead to faster assembly times than abstract AR instructions alone, and that concrete AR instructions lead to greater increases in user confidence than paper-based instructions. The study failed to support the hypothesis that abstract AR elements are more suitable for part identification than concrete AR elements. Finally, the study results and hypothesis conclusions are used to suggest future work regarding interface element design for AR assembly applications

Multiscale motion saliency for keyframe extraction from motion capture sequences

Cataloged from PDF version of article.Motion capture is an increasingly popular animation technique; however data acquired by motion capture can become substantial. This makes it difficult to use motion capture data in a number of applications, such as motion editing, motion understanding, automatic motion summarization, motion thumbnail generation, or motion database search and retrieval. To overcome this limitation, we propose an automatic approach to extract keyframes from a motion capture sequence. We treat the input sequence as motion curves, and obtain the most salient parts of these curves using a new proposed metric, called 'motion saliency'. We select the curves to be analysed by a dimension reduction technique, Principal Component Analysis (PCA). We then apply frame reduction techniques to extract the most important frames as keyframes of the motion. With this approach, around 8% of the frames are selected to be keyframes for motion capture sequences. Copyright (C) 2010 John Wiley & Sons, Ltd

Space-time sketching of character animation

International audienceWe present a space-time abstraction for the sketch-based design of character animation. It allows animators to draft a full coordinated motion using a single stroke called the space-time curve (STC). From the STC we compute a dynamic line of action (DLOA) that drives the motion of a 3D character through projective constraints. Our dynamic models for the line's motion are entirely geometric, require no pre-existing data, and allow full artistic control. The resulting DLOA can be refined by over-sketching strokes along the space-time curve, or by composing another DLOA on top leading to control over complex motions with few strokes. Additionally , the resulting dynamic line of action can be applied to arbitrary body parts or characters. To match a 3D character to the 2D line over time, we introduce a robust matching algorithm based on closed-form solutions, yielding a tight match while allowing squash and stretch of the character's skeleton. Our experiments show that space-time sketching has the potential of bringing animation design within the reach of beginners while saving time for skilled artists

Investigating User Experience Using Gesture-based and Immersive-based Interfaces on Animation Learners

Creating animation is a very exciting activity. However, the long and laborious process can be extremely challenging. Keyframe animation is a complex technique that takes a long time to complete, as the procedure involves changing the poses of characters through modifying the time and space of an action, called frame-by-frame animation. This involves the laborious, repetitive process of constantly reviewing results of the animation in order to make sure the movement-timing is accurate. A new approach to animation is required in order to provide a more intuitive animating experience. With the evolution of interaction design and the Natural User Interface (NUI) becoming widespread in recent years, a NUI-based animation system is expected to allow better usability and efficiency that would benefit animation. This thesis investigates the effectiveness of gesture-based and immersive-based interfaces as part of animation systems. A practice-based element of this research is a prototype of the hand gesture interface, which was created based on experiences from reflective practices. An experimental design is employed to investigate the usability and efficiency of gesture-based and immersive-based interfaces in comparison to the conventional GUI/WIMP interface application. The findings showed that gesture-based and immersive-based interfaces are able to attract animators in terms of the efficiency of the system. However, there was no difference in their preference for usability with the two interfaces. Most of our participants are pleasant with NUI interfaces and new technologies used in the animation process, but for detailed work and taking control of the application, the conventional GUI/WIMP is preferable. Despite the awkwardness of devising gesture-based and immersive-based interfaces for animation, the concept of the system showed potential for a faster animation process, an enjoyable learning system, and stimulating interest in a kinaesthetic learning experience

Robust Motion In-betweening

In this work we present a novel, robust transition generation technique that can serve as a new tool for 3D animators, based on adversarial recurrent neural networks. The system synthesizes high-quality motions that use temporally-sparse keyframes as animation constraints. This is reminiscent of the job of in-betweening in traditional animation pipelines, in which an animator draws motion frames between provided keyframes. We first show that a state-of-the-art motion prediction model cannot be easily converted into a robust transition generator when only adding conditioning information about future keyframes. To solve this problem, we then propose two novel additive embedding modifiers that are applied at each timestep to latent representations encoded inside the network's architecture. One modifier is a time-to-arrival embedding that allows variations of the transition length with a single model. The other is a scheduled target noise vector that allows the system to be robust to target distortions and to sample different transitions given fixed keyframes. To qualitatively evaluate our method, we present a custom MotionBuilder plugin that uses our trained model to perform in-betweening in production scenarios. To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3.6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation. We are releasing this new dataset along with this work, with accompanying code for reproducing our baseline results.Comment: Published at SIGGRAPH 202

Enhancing Expressiveness of Speech through Animated Avatars for Instant Messaging and Mobile Phones

This thesis aims to create a chat program that allows users to communicate via an animated avatar that provides believable lip-synchronization and expressive emotion. Currently many avatars do not attempt to do lip-synchronization. Those that do are not well synchronized and have little or no emotional expression. Most avatars with lip synch use realistic looking 3D models or stylized rendering of complex models. This work utilizes images rendered in a cartoon style and lip-synchronization rules based on traditional animation. The cartoon style, as opposed to a more realistic look, makes the mouth motion more believable and the characters more appealing. The cartoon look and image-based animation (as opposed to a graphic model animated through manipulation of a skeleton or wireframe) also allows for fewer key frames resulting in faster speed with more room for expressiveness. When text is entered into the program, the Festival Text-to-Speech engine creates a speech file and extracts phoneme and phoneme duration data. Believable and fluid lip-synchronization is then achieved by means of a number of phoneme-to-image rules. Alternatively, phoneme and phoneme duration data can be obtained for speech dictated into a microphone using Microsoft SAPI and the CSLU Toolkit. Once lip synchronization has been completed, rules for non-verbal animation are added. Emotions are appended to the animation of speech in two ways: automatically, by recognition of key words and punctuation, or deliberately, by user-defined tags. Additionally, rules are defined for idle-time animation. Preliminary results indicate that the animated avatar program offers an improvement over currently available software. It aids in the understandability of speech, combines easily recognizable and expressive emotions with speech, and successfully enhances overall enjoyment of the chat experience. Applications for the program include use in cell phones for the deaf or hearing impaired, instant messaging, video conferencing, instructional software, and speech and animation synthesis

Detail-Preserving Controllable Deformation from Sparse Examples

published_or_final_versio