Improving automatic rigging for 3D humanoid characters

In the field of computer animation the process of creating an animated character is usually a long and tedious task. An animation character is usually efined by a 3D mesh (a set of triangles in the space) that gives its external appearance or shape to the character. It also used to have an inner structure, the skeleton. When a skeleton is associated to a character mesh, this association is called skeleton binding, and a skeleton bound to a character mesh is an animation rig. Rigging from scratch a character can be a very boring process. The definition and creation of a centered skeleton together with the ’painting’, by an artist,of the influence parameters between the skeleton and the mesh (the skinning) s the most demanding part to achieve an acceptable behavior for a character. This rigging process can be simplified and accelerated using an automatic rigging method. Automatic rigging methods consist in taking as input a 3D mesh, generate a skeleton based in the shape of the original model, bound the input mesh to the generated skeleton, and finally to compute a set of parameters based in a chosen skinning method. The main objective of this thesis is to generate a method for rigging a 3D arbitrary model with minimum user interaction. This can be useful to people without experience in the animation field or to experienced people to accelerate the rigging process from days to hours or minutes depending the needed quality. Having in mind this situation we have designed our method as a set of tools that can be applied to general input models defined by an artist. The contributions made in the development of this thesis can be summarized as: • Generation of an animation Rig: Having an arbitrary closed mesh we have implemented a thinning method to create first an unrefined geometry skeleton that captures the topology and pose of the input character. Using this geometric skeleton as starting point we use a refining method that creates an adjusted logic skeleton based in a template, or may be defined by the user, that is compatible with the current animation formats. The output logic skeleton is specific for each character, and it is bounded to the input mesh to create an animation rig. • Skinning: Having defined an animation rig for an arbitrary mesh we have developed an improved skinning method; this method is based on the Linear Blend Skinning(LBS) algorithm. Our contributions in the skinning field can be sub-divided in: – We propose a segmentation method that works as the core element in a weight assigning algorithm and a skinning lgorithm, we also have developed an automatic algorithm to compute the skin weights of the LBS Skinning of a rigged polygonal mesh. – Our proposed skinning algorithm uses as base the features of the LBS Skinning. The main purpose of the developed algorithm is to solve the well-known ”candy wrap” artifact; that produces a substantial loss of volume when a link of an animation skeleton is rotated over its own axis. We have compared our results with the most important methods in the skinning field, such as Dual Quaternion Skinning (DQS) and LBS, achieving a better performance over DQS and an improvement in quality over LBS. • Animation tools: We have developed a set of Autodesk Maya commands that works together as rig tool, using our previous proposed methods. • Animation loader: Moreover, an animation loader tool has been implemented, that allows the user to load animations from a skeleton with different structure to a rigged 3D model. The contributions previously described has been published in 3 research papers, the first two were presented in international congresses and the third one was acepted for its publication in an JCR indexed journal.En el campo de la animación por computadora el proceso de crear un personaje de animación es comúnmente una tarea larga y tediosa. Un personaje de animación está definido usualmente por una malla tridimensional (un conjunto de triángulos en el espacio) que le dan su apariencia externa y forma al personaje. Es igualmente común que este tenga una estructura interna, un esqueleto de animación. Cuando un esqueleto esta asociado con una malla tridimensional, a esta asociación se le llama ligado de esqueleto, y un esqueleto ligado a la mallade un personaje es conocido en inglés como "animation rig" (el conjunto de elementos necesarios, que unidos sirven para animar un personaje). Hacer el rigging desde cero de un personaje puede ser un proceso muy tedioso. La definición y creación de un esqueleto centrado en la malla junto con el "pintado" por medio de un artista de los parámetros de influencia entre el esqueleto y la malla 3D (lo que se conoce como skinning) es la parte mas demandante para alcanzar un compartimiento aceptable al deformase (moverse) la malla de un personaje. Los métodos de rigging automáticos consisten en tomar una malla tridimensional como elemento de entrada, generar un esqueleto basado en la forma del modelo original, ligar la malla de entrada al esqueleto generado y finamente calcular el conjunto de parámetros utilizados por el método de skinning elegido. El principal objetivo de esta tesis es el generar un método de rigging para un modelo tridimensional arbitrario con una interacción mínima del usuario. Este método puede ser útil para gente con poca experiencia en el campo de la animación, o para gente experimentada que quiera acelerar el proceso de rigging de días a horas o inclusive minutos, dependiendo de la calidad requerida. Teniendo en mente esta situación, hemos diseñado nuestro método como un conjunto de herramientas las cuales pueden ser aplicadas a modelos de entrada generados por cualquier artista. Las contribuciones hechas en el desarrollo de esta tesis pueden resumirse a: -Generación de un rig de animación: Teniendo una malla cerrada cualquiera, hemos implementado un método para crear primero un esqueleto geométrico sin refinar, el cual capture la pose y la topología del personaje usado como elemento de entrada. Tomando este esqueleto geométrico como punto de partida usamos un método de refinado que crea un "esqueleto lógico" adaptado a la forma del geométrico basándonos en una plantilla definida por el usuario o previamente definida, que sea compatible con los formatos actuales de animación. El esqueleto lógico generado será especifico para cada personaje, y esta ligado a la malla de entrada para así crear un rig de animación. - Skinning: Teniendo definido un rig de animación para una malla de entrada arbitraria, hemos desarrollado un método mejorado de skinning, este método sera basado en el algoritmo "Linear Blending Skinnig" (algoritmo de skinning por combinación lineal, LBS por sus siglas en inglés). Nuestras contribuciones en el campo del skinnig son: - Proponemos un nuevo método de segmentación de mallas que sea la parte medular para algoritmos de asignación automática de pesos y de skinning, también hemos desarrollado un algoritmo automático que calcule los pesos utilizados por el algoritmo LBS para una malla poligonal que tenga un rig de animación. - Nuestro algoritmo de skinning propuesto usará como base las características del algoritmo LBS. El principal propósito del algoritmo desarrollado es el solucionar el defecto conocido como "envoltura de caramelo" (candy wrapper artifact), que produce una substancial perdida de volumen al rotar una de las articulaciones del esqueleto de animación sobre su propio eje. Nuestros resultados son comparados con los métodos mas importantes en el campo del skinning tal como Cuaterniones Duales (Dual Quaternions Skinning, DQS) y LBS, alcanzando un mejor desempeño que DQS y una mejora importante sobre LBSPostprint (published version

Automatic skeletonization and skin attachment for realistic character animation.

The realism of character animation is associated with a number of tasks ranging from modelling, skin defonnation, motion generation to rendering. In this research we are concerned with two of them: skeletonization and weight assignment for skin deformation. The fonner is to generate a skeleton, which is placed within the character model and links the motion data to the skin shape of the character. The latter assists the modelling of realistic skin shape when a character is in motion. In the current animation production practice, the task of skeletonization is primarily undertaken by hand, i.e. the animator produces an appropriate skeleton and binds it with the skin model of a character. This is inevitably very time-consuming and costs a lot of labour. In order to improve this issue, in this thesis we present an automatic skeletonization framework. It aims at producing high-quality animatible skeletons without heavy human involvement while allowing the animator to maintain the overall control of the process. In the literature, the tenn skeletonization can have different meanings. Most existing research on skeletonization is in the remit of CAD (Computer Aided Design). Although existing research is of significant reference value to animation, their downside is the skeleton generated is either not appropriate for the particular needs of animation, or the methods are computationally expensive. Although some purpose-build animation skeleton generation techniques exist, unfortunately they rely on complicated post-processing procedures, such as thinning and pruning, which again can be undesirable. The proposed skeletonization framework makes use of a new geometric entity known as the 3D silhouette that is an ordinary silhouette with its depth information recorded. We extract a curve skeleton from two 3D silhouettes of a character detected from its two perpendicular projections. The skeletal joints are identified by down sampling the curve skeleton, leading to the generation of the final animation skeleton. The efficiency and quality are major performance indicators in animation skeleton generation. Our framework achieves the former by providing a 2D solution to the 3D skeletonization problem. Reducing in dimensions brings much faster performances. Experiments and comparisons are carried out to demonstrate the computational simplicity. Its accuracy is also verified via these experiments and comparisons. To link a skeleton to the skin, accordingly we present a skin attachment framework aiming at automatic and reasonable weight distribution. It differs from the conventional algorithms in taking topological information into account during weight computation. An effective range is defined for a joint. Skin vertices located outside the effective range will not be affected by this joint. By this means, we provide a solution to remove the influence of a topologically distant, hence highly likely irrelevant joint on a vertex. A user-defined parameter is also provided in this algorithm, which allows different deformation effects to be obtained according to user's needs. Experiments and comparisons prove that the presented framework results in weight distribution of good quality. Thus it frees animators from tedious manual weight editing. Furthermore, it is flexible to be used with various deformation algorithms

Embedded Implicit Stand-ins for Animated Meshes: a Case of Hybrid Modelling

In this paper we address shape modelling problems, encountered in computer animation and computer games development that are difficult to solve just using polygonal meshes. Our approach is based on a hybrid modelling concept that combines polygonal meshes with implicit surfaces. A hybrid model consists of an animated polygonal mesh and an approximation of this mesh by a convolution surface stand-in that is embedded within it or is attached to it. The motions of both objects are synchronised using a rigging skeleton. This approach is used to model the interaction between an animated mesh object and a viscoelastic substance, normally modelled in implicit form. The adhesive behaviour of the viscous object is modelled using geometric blending operations on the corresponding implicit surfaces. Another application of this approach is the creation of metamorphosing implicit surface parts that are attached to an animated mesh. A prototype implementation of the proposed approach and several examples of modelling and animation with near real-time preview times are presented

CTSN: Predicting Cloth Deformation for Skeleton-based Characters with a Two-stream Skinning Network

We present a novel learning method to predict the cloth deformation for skeleton-based characters with a two-stream network. The characters processed in our approach are not limited to humans, and can be other skeletal-based representations of non-human targets such as fish or pets. We use a novel network architecture which consists of skeleton-based and mesh-based residual networks to learn the coarse and wrinkle features as the overall residual from the template cloth mesh. Our network is used to predict the deformation for loose or tight-fitting clothing or dresses. We ensure that the memory footprint of our network is low, and thereby result in reduced storage and computational requirements. In practice, our prediction for a single cloth mesh for the skeleton-based character takes about 7 milliseconds on an NVIDIA GeForce RTX 3090 GPU. Compared with prior methods, our network can generate fine deformation results with details and wrinkles.Comment: 13 page

SPA: Sparse Photorealistic Animation using a single RGB-D camera

Photorealistic animation is a desirable technique for computer games and movie production. We propose a new method to synthesize plausible videos of human actors with new motions using a single cheap RGB-D camera. A small database is captured in a usual office environment, which happens only once for synthesizing different motions. We propose a markerless performance capture method using sparse deformation to obtain the geometry and pose of the actor for each time instance in the database. Then, we synthesize an animation video of the actor performing the new motion that is defined by the user. An adaptive model-guided texture synthesis method based on weighted low-rank matrix completion is proposed to be less sensitive to noise and outliers, which enables us to easily create photorealistic animation videos with new motions that are different from the motions in the database. Experimental results on the public dataset and our captured dataset have verified the effectiveness of the proposed method

Comparing and Evaluating Real Time Character Engines for Virtual Environments

As animated characters increasingly become vital parts of virtual environments, then the engines that drive these characters increasingly become vital parts of virtual environment software. This paper gives an overview of the state of the art in character engines, and proposes a taxonomy of the features that are commonly found in them. This taxonomy can be used as a tool for comparison and evaluation of different engines. In order to demonstrate this we use it to compare three engines. The first is Cal3D, the most commonly used open source engine. We also introduce two engines created by the authors, Piavca and HALCA. The paper ends with a brief discussion of some other popular engines