Animated surfaces in physically-based simulation

Physics-based animation has become a ubiquitous element in all application areas of computer animation, especially in the entertainment sector. Animation and feature films, video games, and advertisement contain visual effects using physically-based simulation that blend in seamlessly with animated or live-action productions. When simulating deformable materials and fluids, especially liquids, objects are usually represented by animated surfaces. The visual quality of these surfaces not only depends on the actual properties of the surface itself but also on its generation and relation to the underlying simulation. This thesis focuses on surfaces of cloth simulations and fluid simulations based on Smoothed Particle Hydrodynamics (SPH), and contributes to improving the creation of animations by specifying surface shapes, modeling contact of surfaces, and evaluating surface effects of fluids. In many applications, there is a reference given for a surface animation in terms of its shape. Matching a given reference with a simulation is a challenging task and similarity is often determined by visual inspection. The first part of this thesis presents a signature for cloth animations that captures characteristic shapes and their temporal evolution. It combines geometric features with physical properties to represent accurately the typical deformation behavior. The signature enables calculating similarities between animations and is applied to retrieve cloth animations from collections by example. Interactions between particle-based fluids and deformable objects are usually modeled by sampling the deformable objects with particles. When interacting with cloth, however, this would require resampling the surface at large planar deformations and the thickness of cloth would be bound to the particle size. This problem is addressed in this thesis by presenting a two-way coupling technique for cloth and fluids based on the simulation mesh of the textile. It allows robust contact handling and intuitive control of boundary conditions. Further, a solution for intersection-free fluid surface reconstruction at contact with thin flexible objects is presented. The visual quality of particle-based fluid animation highly depends on the properties of the reconstructed surface. An important aspect of the reconstruction method is that it accurately represents the underlying simulation. This thesis presents an evaluation of surfaces at interfaces of SPH simulations incorporating the connection to the simulation model. A typical approach in computer graphics is compared to surface reconstruction used in material sciences. The behavior of free surfaces in fluid animations is highly influenced by surface tension. This thesis presents an evaluation of three types of surface tension models in combination with different pressure force models for SPH to identify the individual characteristics of these models. Systematic tests using a set of benchmark scenes are performed to reveal strengths and weaknesses, and possible areas of applications.Physikalisch basierte Animationen sind ein allgegenwärtiger Teil in jeglichen Anwendungsbereichen der Computeranimation, insbesondere dem Unterhaltungssektor. Animations- und Spielfilme, Videospiele und Werbung enthalten visuelle Effekte unter Verwendung von physikalisch basierter Simulation, die sich nahtlos in Animations- oder Realfilme einfügen. Bei der Simulation von deformierbaren Materialien und Fluiden, insbesondere Flüssigkeiten, werden die Objekte gewöhnlich durch animierte Oberflächen dargestellt. Die visuelle Qualität dieser Oberflächen hängt nicht nur von den Eigenschaften der Fläche selbst ab, sondern auch von deren Erstellung und der Verbindung zu der zugrundeliegenden Simulation. Diese Dissertation widmet sich Oberflächen von Textil- und Fluidsimulationen mit der Methode der Smoothed Particle Hydrodynamics (SPH) und leistet einen Beitrag zur Verbesserung der Erstellung von Animationen durch die Beschreibung von Oberflächenformen, der Modellierung von Kontakt von Oberflächen und der Evaluierung von Oberflächeneffekten von Fluiden. In vielen Anwendungen gibt es eine Referenz für eine Oberflächenanimation, die ihre Form beschreibt. Das Abgleichen einer Referenz mit einer Simulation ist eine große Herausforderung und die Ähnlichkeit wird häufig visuell überprüft. Im ersten Teil der Dissertation wird eine Signatur für Textilanimationen vorgestellt, die charakteristische Formen und ihre zeitliche Veränderung erfasst. Sie ist eine Kombination aus geometrischen Merkmalen und physikalischen Eigenschaften, um das typische Deformationsverhalten genau zu repräsentieren. Die Signatur erlaubt es, Ähnlichkeiten zwischen Animationen zu berechnen, und wird angewendet, um Textilanimationen aus Kollektionen anhand eines Beispiels aufzufinden. Interaktionen zwischen partikelbasierten Fluiden und deformierbaren Objekten werden gewöhnlich durch das Abtasten des deformierbaren Objekts mit Partikeln modelliert. Bei der Interaktion mit Textilien würde dies jedoch ein neues Abtasten bei großen planaren Deformation erfordern und die Stärke des Textils wäre an die Partikelgröße gebunden. Mit diesem Problem befasst sich diese Dissertation und stellt eine Technik für die wechselseitige Kopplung zwischen Textilien und Fluiden vor, die auf dem Simulationsnetz des Textils beruht. Diese erlaubt eine robuste Kontaktbehandlung und intuitive Kontrolle von Randbedingungen. Des Weiteren wird ein Lösungsansatz für eine durchdringungsfreie Oberflächenrekonstruktion beim Kontakt mit dünnen flexiblen Objekten präsentiert. Die visuelle Qualität von partikelbasierten Fluidanimationen hängt stark von den Eigenschaften der rekonstruierten Oberfläche ab. Wichtig bei Rekonstruktionsmethoden ist, dass sie die zugrundeliegende Simulation genau repräsentieren. Die Dissertation präsentiert eine Evaluierung von Oberflächen an Grenzflächen, die den Zusammenhang zum Simulationsmodell miteinbezieht. Ein typischer Ansatz aus der Computergrafik wird mit der Oberflächenrekonstruktion in der Werkstoffkunde verglichen. Das Verhalten von freien Oberflächen in Fluidanimationen wird stark von der Oberflächenspannung beeinflusst. In dieser Dissertation wird eine Evaluierung von drei Oberflächenspannungsmodellen in Kombination mit verschiedenen Druckmodellen für SPH präsentiert, um die Charakteristika der jeweiligen Modelle zu identifizieren. Es werden systematische Tests mit Hilfe von Benchmark-Tests durchgeführt, um Stärken, Schwächen und mögliche Anwendungsbereiche deutlich zu machen

Steklov Spectral Geometry for Extrinsic Shape Analysis

We propose using the Dirichlet-to-Neumann operator as an extrinsic alternative to the Laplacian for spectral geometry processing and shape analysis. Intrinsic approaches, usually based on the Laplace-Beltrami operator, cannot capture the spatial embedding of a shape up to rigid motion, and many previous extrinsic methods lack theoretical justification. Instead, we consider the Steklov eigenvalue problem, computing the spectrum of the Dirichlet-to-Neumann operator of a surface bounding a volume. A remarkable property of this operator is that it completely encodes volumetric geometry. We use the boundary element method (BEM) to discretize the operator, accelerated by hierarchical numerical schemes and preconditioning; this pipeline allows us to solve eigenvalue and linear problems on large-scale meshes despite the density of the Dirichlet-to-Neumann discretization. We further demonstrate that our operators naturally fit into existing frameworks for geometry processing, making a shift from intrinsic to extrinsic geometry as simple as substituting the Laplace-Beltrami operator with the Dirichlet-to-Neumann operator.Comment: Additional experiments adde

Non-interactive modeling tools and support environment for procedural geometry generation

This research examines procedural modeling in the eld of computer graphics. Procedural modeling automates the generation of objects by representing models as procedures that provide a description of the process required to create the model. The problem we solve with this research is the creation of a procedural modeling environment that consists of a procedural modeling language and a set of non-interactive modeling tools. A goal of this research is to provide comparisons between 3D manual modeling and procedural modeling, which focus on the modeling strategies, tools and model representations used by each modeling paradigm. A procedural modeling language is presented that has the same facilities and features of existing procedural modeling languages. In addition, features such as caching and a pseudorandom number generator is included, demonstrating the advantages of a procedural modeling paradigm. The non-interactive tools created within the procedural modeling framework are selection, extrusion, subdivision, curve shaping and stitching. In order to demonstrate the usefulness of the procedural modeling framework, human and furniture models are created using this procedural modeling environment. Various techniques are presented to generate these objects, and may be used to create a variety of other models. A detailed discussion of each technique is provided. Six experiments are conducted to test the support of the procedural modeling benets provided by this non- interactive modeling environment. The experiments test, namely parameterisation, re-usability, base-shape independence, model complexity, the generation of reproducible random numbers and caching. We prove that a number of distinct models can be generated from a single procedure through the use parameterisation. Modeling procedures and sub-procedures are re-usable and can be applied to different models. Procedures can be base-shape independent. The level of complexity of a model can be increased by repeatedly applying geometry to the model. The pseudo-random number generator is capable of generating reproducible random numbers. The caching facility reduces the time required to generate a model that uses repetitive geometry

MONOCULAR POSE ESTIMATION AND SHAPE RECONSTRUCTION OF QUASI-ARTICULATED OBJECTS WITH CONSUMER DEPTH CAMERA

Quasi-articulated objects, such as human beings, are among the most commonly seen objects in our daily lives. Extensive research have been dedicated to 3D shape reconstruction and motion analysis for this type of objects for decades. A major motivation is their wide applications, such as in entertainment, surveillance and health care. Most of existing studies relied on one or more regular video cameras. In recent years, commodity depth sensors have become more and more widely available. The geometric measurements delivered by the depth sensors provide significantly valuable information for these tasks. In this dissertation, we propose three algorithms for monocular pose estimation and shape reconstruction of quasi-articulated objects using a single commodity depth sensor. These three algorithms achieve shape reconstruction with increasing levels of granularity and personalization. We then further develop a method for highly detailed shape reconstruction based on our pose estimation techniques. Our first algorithm takes advantage of a motion database acquired with an active marker-based motion capture system. This method combines pose detection through nearest neighbor search with pose refinement via non-rigid point cloud registration. It is capable of accommodating different body sizes and achieves more than twice higher accuracy compared to a previous state of the art on a publicly available dataset. The above algorithm performs frame by frame estimation and therefore is less prone to tracking failure. Nonetheless, it does not guarantee temporal consistent of the both the skeletal structure and the shape and could be problematic for some applications. To address this problem, we develop a real-time model-based approach for quasi-articulated pose and 3D shape estimation based on Iterative Closest Point (ICP) principal with several novel constraints that are critical for monocular scenario. In this algorithm, we further propose a novel method for automatic body size estimation that enables its capability to accommodate different subjects. Due to the local search nature, the ICP-based method could be trapped to local minima in the case of some complex and fast motions. To address this issue, we explore the potential of using statistical model for soft point correspondences association. Towards this end, we propose a unified framework based on Gaussian Mixture Model for joint pose and shape estimation of quasi-articulated objects. This method achieves state-of-the-art performance on various publicly available datasets. Based on our pose estimation techniques, we then develop a novel framework that achieves highly detailed shape reconstruction by only requiring the user to move naturally in front of a single depth sensor. Our experiments demonstrate reconstructed shapes with rich geometric details for various subjects with different apparels. Last but not the least, we explore the applicability of our method on two real-world applications. First of all, we combine our ICP-base method with cloth simulation techniques for Virtual Try-on. Our system delivers the first promising 3D-based virtual clothing system. Secondly, we explore the possibility to extend our pose estimation algorithms to assist physical therapist to identify their patients’ movement dysfunctions that are related to injuries. Our preliminary experiments have demonstrated promising results by comparison with the gold standard active marker-based commercial system. Throughout the dissertation, we develop various state-of-the-art algorithms for pose estimation and shape reconstruction of quasi-articulated objects by leveraging the geometric information from depth sensors. We also demonstrate their great potentials for different real-world applications

Human Shape from Silhouettes Using Generative HKS Descriptors and Cross-Modal Neural Networks

In this work, we present a novel method for capturing human body shape from a single scaled silhouette. We combine deep correlated features capturing different 2D views, and embedding spaces based on 3D cues in a novel convolutional neural network (CNN) based architecture. We first train a CNN to find a richer body shape representation space from pose invariant 3D human shape descriptors. Then, we learn a mapping from silhouettes to this representation space, with the help of a novel architecture that exploits the correlation of multi-view data during training time, to improve prediction at test time. We extensively validate our results on synthetic and real data, demonstrating significant improvements in accuracy as compared to the state-of-the-art, and providing a practical system for detailed human body measurements from a single image

Semantics for virtual humans

Population of Virtual Worlds with Virtual Humans is increasing rapidly by people who want to create a virtual life parallel to the real one (i.e. Second Life). The evolution of technology is smoothly providing the necessary elements to increase realism within these virtual worlds by creating believable Virtual Humans. However, creating the amount of resources needed to succeed this believability is a difficult task, mainly because of the complexity of the creation process of Virtual Humans. Even though there are many existing available resources, their reusability is difficult because there is not enough information provided to evaluate if a model contains the desired characteristics to be reused. Additionally, the knowledge involved in the creation of Virtual Humans is not well known, nor well disseminated. There are several different creation techniques, different software components, and several processes to carry out before having a Virtual Human capable of populating a virtual environment. The creation of Virtual Humans involves: a geometrical representation with an internal control structure, the motion synthesis with different animation techniques, higher level controllers and descriptors to simulate human-like behavior such individuality, cognition, interaction capabilities, etc. All these processes require the expertise from different fields of knowledge such as mathematics, artificial intelligence, computer graphics, design, etc. Furthermore, there is neither common framework nor common understanding of how elements involved in the creation, development, and interaction of Virtual Humans features are done. Therefore, there is a need for describing (1) existing resources, (2) Virtual Human's composition and features, (3) a creation pipeline and (4) the different levels/fields of knowledge comprehended. This thesis presents an explicit representation of the Virtual Humans and their features to provide a conceptual framework that will interest to all people involved in the creation and development of these characters. This dissertation focuses in a semantic description of Virtual Humans. The creation of a semantic description involves gathering related knowledge, agreement among experts in the definition of concepts, validation of the ontology design, etc. In this dissertation all these procedures are presented, and an Ontology for Virtual Humans is described in detail together with the validations that conducted to the resulted ontology. The goal of creating such ontology is to promote reusability of existing resources; to create a shared knowledge of the creation and composition of Virtual Humans; and to support new research of the fields involved in the development of believable Virtual Humans and virtual environments. Finally, this thesis presents several developments that aim to demonstrate the ontology usability and reusability. These developments serve particularly to support the research on specialized knowledge of Virtual Humans, the population of virtual environments, and improve the believability of these characters

From scans to models: Registration of 3D human shapes exploiting texture information

New scanning technologies are increasing the importance of 3D mesh data, and of algorithms that can reliably register meshes obtained from multiple scans. Surface registration is important e.g. for building full 3D models from partial scans, identifying and tracking objects in a 3D scene, creating statistical shape models. Human body registration is particularly important for many applications, ranging from biomedicine and robotics to the production of movies and video games; but obtaining accurate and reliable registrations is challenging, given the articulated, non-rigidly deformable structure of the human body. In this thesis, we tackle the problem of 3D human body registration. We start by analyzing the current state of the art, and find that: a) most registration techniques rely only on geometric information, which is ambiguous on flat surface areas; b) there is a lack of adequate datasets and benchmarks in the field. We address both issues. Our contribution is threefold. First, we present a model-based registration technique for human meshes that combines geometry and surface texture information to provide highly accurate mesh-to-mesh correspondences. Our approach estimates scene lighting and surface albedo, and uses the albedo to construct a high-resolution textured 3D body model that is brought into registration with multi-camera image data using a robust matching term. Second, by leveraging our technique, we present FAUST (Fine Alignment Using Scan Texture), a novel dataset collecting 300 high-resolution scans of 10 people in a wide range of poses. FAUST is the first dataset providing both real scans and automatically computed, reliable ground-truth correspondences between them. Third, we explore possible uses of our approach in dermatology. By combining our registration technique with a melanocytic lesion segmentation algorithm, we propose a system that automatically detects new or evolving lesions over almost the entire body surface, thus helping dermatologists identify potential melanomas. We conclude this thesis investigating the benefits of using texture information to establish frame-to-frame correspondences in dynamic monocular sequences captured with consumer depth cameras. We outline a novel approach to reconstruct realistic body shape and appearance models from dynamic human performances, and show preliminary results on challenging sequences captured with a Kinect

Multi-View Dynamic Shape Refinement Using Local Temporal Integration

International audienceWe consider 4D shape reconstructions in multi-view environments and investigate how to exploit temporal redundancy for precision refinement. In addition to being beneficial to many dynamic multi-view scenarios this also enables larger scenes where such increased precision can compensate for the reduced spatial resolution per image frame. With precision and scalability in mind, we propose a symmetric (non-causal) local time-window geometric integration scheme over temporal sequences, where shape reconstructions are refined framewise by warping local and reliable geometric regions of neighboring frames to them. This is in contrast to recent comparable approaches targeting a different context with more compact scenes and real-time applications. These usually use a single dense volumetric update space or geometric template, which they causally track and update globally frame by frame, with limitations in scalability for larger scenes and in topology and precision with a template based strategy. Our templateless and local approach is a first step towards temporal shape super-resolution. We show that it improves reconstruction accuracy by considering multiple frames. To this purpose, and in addition to real data examples, we introduce a multi-camera synthetic dataset that provides ground-truth data for mid-scale dynamic scenes

Rekonstruktion, Analyse und Editierung dynamisch deformierter 3D-Oberflächen

Dynamically deforming 3D surfaces play a major role in computer graphics. However, producing time-varying dynamic geometry at ever increasing detail is a time-consuming and costly process, and so a recent trend is to capture geometry data directly from the real world. In the first part of this thesis, I propose novel approaches for this research area. These approaches capture dense dynamic 3D surfaces from multi-camera systems in a particularly robust and accurate way. This provides highly realistic dynamic surface models for phenomena like moving garments and bulging muscles. However, re-using, editing, or otherwise analyzing dynamic 3D surface data is not yet conveniently possible. To close this gap, the second part of this dissertation develops novel data-driven modeling and animation approaches. I first show a supervised data-driven approach for modeling human muscle deformations that scales to huge datasets and provides fine-scale, anatomically realistic deformations at high quality not attainable by previous methods. I then extend data-driven modeling to the unsupervised case, providing editing tools for a wider set of input data ranging from facial performance capture and full-body motion to muscle and cloth deformation. To this end, I introduce the concepts of sparsity and locality within a mathematical optimization framework. I also explore these concepts for constructing shape-aware functions that are useful for static geometry processing, registration, and localized editing.Dynamisch deformierbare 3D-Oberflächen spielen in der Computergrafik eine zentrale Rolle. Die Erstellung der für Computergrafik-Anwendungen benötigten, hochaufgelösten und zeitlich veränderlichen Oberflächengeometrien ist allerdings äußerst arbeitsintensiv. Aus dieser Problematik heraus hat sich der Trend entwickelt, Oberflächendaten direkt aus Aufnahmen der echten Welt zu erfassen. Dazu nötige 3D-Rekonstruktionsverfahren werden im ersten Teil der Arbeit entwickelt. Die vorgestellten, neuartigen Verfahren erlauben die Erfassung dynamischer 3D-Oberflächen aus Mehrkamera-Aufnahmen bei hoher Verlässlichkeit und Präzision. Auf diese Weise können detaillierte Oberflächenmodelle von Phänomenen wie in Bewegung befindliche Kleidung oder sich anspannende Muskeln erfasst werden. Aber auch die Wiederverwendung, Bearbeitung und Analyse derlei gewonnener 3D-Oberflächendaten ist aktuell noch nicht auf eine einfache Art und Weise möglich. Um diese Lücke zu schließen beschäftigt sich der zweite Teil der Arbeit mit der datengetriebenen Modellierung und Animation. Zunächst wird ein Ansatz für das überwachte Lernen menschlicher Muskel-Deformationen vorgestellt. Dieses neuartige Verfahren ermöglicht eine datengetriebene Modellierung mit besonders umfangreichen Datensätzen und liefert anatomisch-realistische Deformationseffekte. Es übertrifft damit die Genauigkeit früherer Methoden. Im nächsten Teil beschäftigt sich die Dissertation mit dem unüberwachten Lernen aus 3D-Oberflächendaten. Es werden neuartige Werkzeuge vorgestellt, die eine weitreichende Menge an Eingabedaten verarbeiten können, von aufgenommenen Gesichtsanimationen über Ganzkörperbewegungen bis hin zu Muskel- und Kleidungsdeformationen. Um diese Anwendungsbreite zu erreichen stützt sich die Arbeit auf die allgemeinen Konzepte der Spärlichkeit und Lokalität und bettet diese in einen mathematischen Optimierungsansatz ein. Abschließend zeigt die vorliegende Arbeit, wie diese Konzepte auch für die Konstruktion von oberflächen-adaptiven Basisfunktionen übertragen werden können. Dadurch können Anwendungen für die Verarbeitung, Registrierung und Bearbeitung statischer Oberflächenmodelle erschlossen werden