Enhancing Mesh Deformation Realism: Dynamic Mesostructure Detailing and Procedural Microstructure Synthesis

Propomos uma solução para gerar dados de mapas de relevo dinâmicos para simular deformações em superfícies macias, com foco na pele humana. A solução incorpora a simulação de rugas ao nível mesoestrutural e utiliza texturas procedurais para adicionar detalhes de microestrutura estáticos. Oferece flexibilidade além da pele humana, permitindo a geração de padrões que imitam deformações em outros materiais macios, como couro, durante a animação. As soluções existentes para simular rugas e pistas de deformação frequentemente dependem de hardware especializado, que é dispendioso e de difícil acesso. Além disso, depender exclusivamente de dados capturados limita a direção artística e dificulta a adaptação a mudanças. Em contraste, a solução proposta permite a síntese dinâmica de texturas que se adaptam às deformações subjacentes da malha de forma fisicamente plausível. Vários métodos foram explorados para sintetizar rugas diretamente na geometria, mas sofrem de limitações como auto-interseções e maiores requisitos de armazenamento. A intervenção manual de artistas na criação de mapas de rugas e mapas de tensão permite controle, mas pode ser limitada em deformações complexas ou onde maior realismo seja necessário. O nosso trabalho destaca o potencial dos métodos procedimentais para aprimorar a geração de padrões de deformação dinâmica, incluindo rugas, com maior controle criativo e sem depender de dados capturados. A incorporação de padrões procedimentais estáticos melhora o realismo, e a abordagem pode ser estendida além da pele para outros materiais macios.We propose a solution for generating dynamic heightmap data to simulate deformations for soft surfaces, with a focus on human skin. The solution incorporates mesostructure-level wrinkles and utilizes procedural textures to add static microstructure details. It offers flexibility beyond human skin, enabling the generation of patterns mimicking deformations in other soft materials, such as leater, during animation. Existing solutions for simulating wrinkles and deformation cues often rely on specialized hardware, which is costly and not easily accessible. Moreover, relying solely on captured data limits artistic direction and hinders adaptability to changes. In contrast, our proposed solution provides dynamic texture synthesis that adapts to underlying mesh deformations. Various methods have been explored to synthesize wrinkles directly to the geometry, but they suffer from limitations such as self-intersections and increased storage requirements. Manual intervention by artists using wrinkle maps and tension maps provides control but may be limited to the physics-based simulations. Our research presents the potential of procedural methods to enhance the generation of dynamic deformation patterns, including wrinkles, with greater creative control and without reliance on captured data. Incorporating static procedural patterns improves realism, and the approach can be extended to other soft-materials beyond skin

Facial hacking: the twisted logic of electro-facial choreography

This research addresses the development of a computational facial language that enables systematic exploration of the external controlled human face with the aim to identify fundamental electro-facial choreographic patterns. Rewiring the human face to an external digital control system, has sparked a radical new way of thinking about the human facial display. Radical, as facial movement is now rooted in digital instead of neural computation. The human face has become an extension of a digital control system inheriting its characteristics: i.e. temporal accuracy, consistency of execution and high programmability. How do we conceptualize the thinking about the human face as a digital computational display device? What are the implications of the “regime change” from neural to digital? The research addresses these issues within the contextual framework where it also originated, in the practice of hacking. It uses the results oriented methods and strategies of hacking to analyze, explore and contextualize the human facial display as a site for digital computational expression. The contributions of this work include the following. 1) External facial control transgresses the neural performance limitations and enables us to think about facial movement from a digital computational choreographic paradigm. 2) A facial language, the Language of Facial E-motion, that allows systematic computational exploration of possible facial movement patterns. Choreologic probing of dynamic face space has brought about unseen facial movement patterns and has uncovered a latent expressive potential of the facial hardware

Uncovering the specificities of CAD tools for industrial design with design theory – style models for generic singularity

International audienceAccording to some casual observers, computer-aided design (CAD) tools are very similar. These tools are used to design new artifacts in a digital environment; hence, they share typical software components, such as a computing engine and human-machine interface. However, CAD software is dedicated to specific professionals—such as engineers, three-dimensional (3D) artists, and industrial designers (IDs)—who claim that, despite their apparent similarities, CAD tools are so different that they are not substitutable. Moreover, CAD tools do not fully meet the needs of IDs. This paper aims at better characterizing CAD tools by taking into account their underlying design logic, which involves relying on recent advances in design theory. We show that engineering CAD tools are actually modeling tools that design a generic variety of products; 3D artist CAD tools not only design but immediately produce single digital artefacts; and ID CAD tools are neither a mix nor an hybridization of engineering CAD and 3D artist CAD tools but have their own logic, namely to create new conceptual models for a large variety of products, that is, the creation of a unique original style that leads to a generic singularity. Such tools are useful for many creative designers beyond IDs

Material agency and performative dynamics in the practices of media art

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityThis dissertation identifies a strategy of artistic inquiry within contemporary media art practice. It applies the concept of material that acts in an agential capacity, generating performative acts. It argues that the emergent potentials of materials and their interconnectedness with the compositional layers of a work can facilitate modes of effecting change in the artistic system. Through the theoretical investigation of the production processes of physical structures and environments, the thesis focuses on the compositional dynamics within which materials actively perform. It examines how Lars Spuybroek’s architectural design method of Material Machines (2004), and both the tactile potential as well as tactical uses of materials as generators to the formtaking process, might describe an open and active artistic strategy for employing the experimental capacities of such materialization processes. Building on philosophical and conceptual arguments that trace concepts of agency (Bruno Latour’s Actant-Network theory) and enactment (Karen Barad’s concept of intra-acting), the thesis introduces the two installation works ANI_MATE (described as a performative pneumatic stage machine) and ON TRACK (described as a mechanic-robotic installation). These apply the introduced artistic strategies. The analyses of these two artworks traces the particular capacities of the materials involved (respectively, their elasticity or viscosity) to negotiate forces of physical movement, which effect the system to transiently or irreversibly transform. ANI_MATE is a machine that is artist-operated and that explores the relationship between liveanimation procedures and the transformability and flexibility of its material environment. In contrast, ON TRACK’s performative machine ecology removes human agency. The machines act autonomously, giving rise to chance in the artistic system and allowing agency to emerge from the dynamic interconnectivity between materials, parts, and processes, eventually producing an entropic scenario of spilling resources. The thesis concludes that, in the context of a post digital paradigm in-development, such artistic practice offers a new strategy for an emergent aesthetics within contemporary physical-digital performance

Muscle activation mapping of skeletal hand motion: an evolutionary approach.

Creating controlled dynamic character animation consists of mathe- matical modelling of muscles and solving the activation dynamics that form the key to coordination. But biomechanical simulation and control is com- putationally expensive involving complex di erential equations and is not suitable for real-time platforms like games. Performing such computations at every time-step reduces frame rate. Modern games use generic soft- ware packages called physics engines to perform a wide variety of in-game physical e ects. The physics engines are optimized for gaming platforms. Therefore, a physics engine compatible model of anatomical muscles and an alternative control architecture is essential to create biomechanical charac- ters in games. This thesis presents a system that generates muscle activations from captured motion by borrowing principles from biomechanics and neural con- trol. A generic physics engine compliant muscle model primitive is also de- veloped. The muscle model primitive forms the motion actuator and is an integral part of the physical model used in the simulation. This thesis investigates a stochastic solution to create a controller that mimics the neural control system employed in the human body. The control system uses evolutionary neural networks that evolve its weights using genetic algorithms. Examples and guidance often act as templates in muscle training during all stages of human life. Similarly, the neural con- troller attempts to learn muscle coordination through input motion samples. The thesis also explores the objective functions developed that aids in the genetic evolution of the neural network. Character interaction with the game world is still a pre-animated behaviour in most current games. Physically-based procedural hand ani- mation is a step towards autonomous interaction of game characters with the game world. The neural controller and the muscle primitive developed are used to animate a dynamic model of a human hand within a real-time physics engine environment

LEARNING TO RIG CHARACTERS

With the emergence of 3D virtual worlds, 3D social media, and massive online games, the need for diverse, high-quality, animation-ready characters and avatars is greater than ever. To animate characters, artists hand-craft articulation structures, such as animation skeletons and part deformers, which require significant amount of manual and laborious interaction with 2D/3D modeling interfaces. This thesis presents deep learning methods that are able to significantly automate the process of character rigging. First, the thesis introduces RigNet, a method capable of predicting an animation skeleton for an input static 3D shape in the form of a polygon mesh. The predicted skeletons match the animator expectations in joint placement and topology. RigNet also estimates surface skin weights which determine how the mesh is animated given the different skeletal poses. In contrast to prior work that fits pre-defined skeletal templates with hand-tuned objectives, RigNet is able to automatically rig diverse characters, such as humanoids, quadrupeds, toys, birds, with varying articulation structure and geometry. RigNet is based on a deep neural architecture that directly operates on the mesh representation. The architecture is trained on a diverse dataset of rigged models that we mined online and curated. The dataset includes 2.7K polygon meshes, along with their associated skeletons and corresponding skin weights. Second, the thesis introduces Morig, a method that automatically rigs character meshes driven by single-view point cloud streams capturing the motion of performing characters. Compared to RigNet, MoRig\u27s rigging is \emph{motion-aware}: its neural network encodes motion cues from the point clouds into compact feature representations that are informative about the articulated parts of the performing character. These motion-aware features guide the inference of an appropriate skeletal rig for the input mesh. Furthermore, Morig is able to animate the rig according to the captured point cloud motion. Morig can handle diverse characters with different morphologies (e.g., humanoids, quadrupeds, toy characters). It also accounts for occluded regions in the point clouds and mismatches in the part proportions between the input mesh and captured character. Third, the thesis introduces APES, a method that takes as input 2D raster images depicting a small set of poses of a character shown in a sprite sheet, and identifies articulated parts useful for rigging the character. APES uses a combination of neural network inference and integer linear programming to identify a compact set of articulated body parts, e.g. head, torso and limbs, that best reconstruct the input poses. Compared to Morig and RigNet that require a large collection of training models with associated skeletons and skinning weights, APES\u27 neural architecture relies on less effortful supervision from (i) pixel correspondences readily available in existing large cartoon image datasets (e.g., Creative Flow), (ii) a relatively small dataset of 57 cartoon characters segmented into moving parts. Finally, the thesis discusses future research directions related to combining neural rigging with 3D and 4D reconstruction of characters from point cloud data and 2D video as well as automating the process of motion synthesis for 3D characters