Enhancing Mesh Deformation Realism: Dynamic Mesostructure Detailing and Procedural Microstructure Synthesis

Propomos uma solução para gerar dados de mapas de relevo dinâmicos para simular deformações em superfícies macias, com foco na pele humana. A solução incorpora a simulação de rugas ao nível mesoestrutural e utiliza texturas procedurais para adicionar detalhes de microestrutura estáticos. Oferece flexibilidade além da pele humana, permitindo a geração de padrões que imitam deformações em outros materiais macios, como couro, durante a animação. As soluções existentes para simular rugas e pistas de deformação frequentemente dependem de hardware especializado, que é dispendioso e de difícil acesso. Além disso, depender exclusivamente de dados capturados limita a direção artística e dificulta a adaptação a mudanças. Em contraste, a solução proposta permite a síntese dinâmica de texturas que se adaptam às deformações subjacentes da malha de forma fisicamente plausível. Vários métodos foram explorados para sintetizar rugas diretamente na geometria, mas sofrem de limitações como auto-interseções e maiores requisitos de armazenamento. A intervenção manual de artistas na criação de mapas de rugas e mapas de tensão permite controle, mas pode ser limitada em deformações complexas ou onde maior realismo seja necessário. O nosso trabalho destaca o potencial dos métodos procedimentais para aprimorar a geração de padrões de deformação dinâmica, incluindo rugas, com maior controle criativo e sem depender de dados capturados. A incorporação de padrões procedimentais estáticos melhora o realismo, e a abordagem pode ser estendida além da pele para outros materiais macios.We propose a solution for generating dynamic heightmap data to simulate deformations for soft surfaces, with a focus on human skin. The solution incorporates mesostructure-level wrinkles and utilizes procedural textures to add static microstructure details. It offers flexibility beyond human skin, enabling the generation of patterns mimicking deformations in other soft materials, such as leater, during animation. Existing solutions for simulating wrinkles and deformation cues often rely on specialized hardware, which is costly and not easily accessible. Moreover, relying solely on captured data limits artistic direction and hinders adaptability to changes. In contrast, our proposed solution provides dynamic texture synthesis that adapts to underlying mesh deformations. Various methods have been explored to synthesize wrinkles directly to the geometry, but they suffer from limitations such as self-intersections and increased storage requirements. Manual intervention by artists using wrinkle maps and tension maps provides control but may be limited to the physics-based simulations. Our research presents the potential of procedural methods to enhance the generation of dynamic deformation patterns, including wrinkles, with greater creative control and without reliance on captured data. Incorporating static procedural patterns improves realism, and the approach can be extended to other soft-materials beyond skin

FDLS: A Deep Learning Approach to Production Quality, Controllable, and Retargetable Facial Performances

Visual effects commonly requires both the creation of realistic synthetic humans as well as retargeting actors' performances to humanoid characters such as aliens and monsters. Achieving the expressive performances demanded in entertainment requires manipulating complex models with hundreds of parameters. Full creative control requires the freedom to make edits at any stage of the production, which prohibits the use of a fully automatic ``black box'' solution with uninterpretable parameters. On the other hand, producing realistic animation with these sophisticated models is difficult and laborious. This paper describes FDLS (Facial Deep Learning Solver), which is Weta Digital's solution to these challenges. FDLS adopts a coarse-to-fine and human-in-the-loop strategy, allowing a solved performance to be verified and edited at several stages in the solving process. To train FDLS, we first transform the raw motion-captured data into robust graph features. Secondly, based on the observation that the artists typically finalize the jaw pass animation before proceeding to finer detail, we solve for the jaw motion first and predict fine expressions with region-based networks conditioned on the jaw position. Finally, artists can optionally invoke a non-linear finetuning process on top of the FDLS solution to follow the motion-captured virtual markers as closely as possible. FDLS supports editing if needed to improve the results of the deep learning solution and it can handle small daily changes in the actor's face shape. FDLS permits reliable and production-quality performance solving with minimal training and little or no manual effort in many cases, while also allowing the solve to be guided and edited in unusual and difficult cases. The system has been under development for several years and has been used in major movies.Comment: DigiPro '22: The Digital Production Symposiu

Sparse Volumetric Deformation

Volume rendering is becoming increasingly popular as applications require realistic solid shape representations with seamless texture mapping and accurate filtering. However rendering sparse volumetric data is difficult because of the limited memory and processing capabilities of current hardware. To address these limitations, the volumetric information can be stored at progressive resolutions in the hierarchical branches of a tree structure, and sampled according to the region of interest. This means that only a partial region of the full dataset is processed, and therefore massive volumetric scenes can be rendered efficiently. The problem with this approach is that it currently only supports static scenes. This is because it is difficult to accurately deform massive amounts of volume elements and reconstruct the scene hierarchy in real-time. Another problem is that deformation operations distort the shape where more than one volume element tries to occupy the same location, and similarly gaps occur where deformation stretches the elements further than one discrete location. It is also challenging to efficiently support sophisticated deformations at hierarchical resolutions, such as character skinning or physically based animation. These types of deformation are expensive and require a control structure (for example a cage or skeleton) that maps to a set of features to accelerate the deformation process. The problems with this technique are that the varying volume hierarchy reflects different feature sizes, and manipulating the features at the original resolution is too expensive; therefore the control structure must also hierarchically capture features according to the varying volumetric resolution. This thesis investigates the area of deforming and rendering massive amounts of dynamic volumetric content. The proposed approach efficiently deforms hierarchical volume elements without introducing artifacts and supports both ray casting and rasterization renderers. This enables light transport to be modeled both accurately and efficiently with applications in the fields of real-time rendering and computer animation. Sophisticated volumetric deformation, including character animation, is also supported in real-time. This is achieved by automatically generating a control skeleton which is mapped to the varying feature resolution of the volume hierarchy. The output deformations are demonstrated in massive dynamic volumetric scenes

All-Women Initiatives in Art and Technology 1986-2020. Atenea: Mentoring and Networking Project in Steam

[ES] La presente tesis doctoral es una exploración sobre las iniciativas auto-organizadas de mujeres que han surgido en la intersección del arte y la tecnología a nivel internacional. Estas iniciativas comunitarias han ido apareciendo en lugares donde existe una escena de arte y nuevos medios, la cual es generalmente de dominación masculina y baja en diversidad. Este trabajo de investigación es un mapeo de este fenómeno tecno-social y artístico, tratando de identificar las primeras prácticas. A través de entrevistas, encuestas y métodos de investigación etnográfica, esta tesis aspira a comprender el contexto que ha dado lugar a semejantes iniciativas a lo largo de tres décadas y media, examinando las diferentes tipologías por los formatos, extensión, espacio y estructura que éstas han adoptado para afrontar la situación de las mujeres en estas áreas. Existe un vació de referencias sobre formas organizativas y de comisariado en las áreas de arte y tecnología. Además, la literatura general señala las subculturas digitales de los noventa, tales como el ciberfeminismo y el movimiento DIY, como los primeros y principales referentes para las alianzas de mujeres y el activismo translocal en la era digital. Sin embargo, las estrategias de colaboración y segregación en áreas de arte y tecnología no han sido analizadas desde los estudios feministas desde una perspectiva histórica en profundidad. Por lo tanto, esta tesis establece la relación entre dichas iniciativas con la historia de los grupos de mujeres artistas y espacios de arte alternativos, la historia de las mujeres en el arte de nuevos medios y la cultura de redes. Las nuevas tecnologías de la información y comunicación han permitido nuevas formas de comunicación y conectividad transfronterizas; sin embargo, este trabajo investiga en qué medida estas iniciativas dependen de las TIC para el establecimiento de su comunidad, comparado con los formatos analógicos. Con los resultados, esta tesis aspira a convertirse en una fuente de referencias que enriquezca la escasa bibliografía de la temática, proporcionando una lista de formas organizativas y de comisariado, así como las practicas más tempranas nunca publicadas, y en conjunto contribuir a la historiografía actual sobre mujeres. Por último, a la vez que se comenta la situación actual de las mujeres en las áreas de arte y tecnología, este trabajo presenta Atenea, un proyecto de mentorización y networking para mujeres en las artes y carreras STEM. Atenea está compuesto de un programa de actividades con diferentes actividades organizativas y de comisariado, así como una plataforma de networking. Adicionalmente, el proyecto tiene un programa enfocado en transmitir conocimientos STEM a niñas a través de prácticas artísticas con tecnología de la mano de mujeres artistas nacionales como mentoras.[CAT] La present tesi doctoral és una exploració sobre les iniciatives autoorganitzades de dones que han sorgit en la intersecció de l'art i la tecnologia a nivell internacional. Aquestes iniciatives comunitàries han anat apareixent en llocs on hi ha una escena d'art i nous mitjans, la qual és generalment de dominació masculina i baixa en diversitat. Aquest treball de recerca és un mapeig d'aquest fenomen tecno-social i artístic, tractant d'identificar les primeres pràctiques. A través d'entrevistes, enquestes i mètodes d'investigació etnogràfica, aquesta tesi aspira a comprendre el context que ha donat lloc a semblants iniciatives al llarg de tres dècades i mig, examinant les diferents tipologies pels formats, extensió, espai i estructura que aquestes han adoptat per afrontar la situació de les dones en aquestes àrees. Hi ha un buit de referències sobre formes organitzatives i de comissariat en les àrees d'art i tecnologia. A més, la literatura general assenyala les subcultures digitals dels noranta, com ara el ciberfeminisme i el moviment DIY, com els primers i principals referents per a les aliances de dones i l'activisme translocal en l'era digital. No obstant això, les estratègies de col·laboració i segregació en àrees d'art i tecnologia no han estat analitzades des dels estudis feministes des d'una perspectiva històrica en profunditat. Per tant, aquesta tesi estableix la relació entre aquestes iniciatives amb la història dels grups de dones artistes i espais d'art alternatius, la història de les dones en l'art de nous mitjans i la cultura de xarxes. Les noves tecnologies de la informació i comunicació han permès noves formes de comunicació i connectivitat transfrontereres; però, aquest treball investiga en quina mesura aquestes iniciatives depenen de les TIC per a l'establiment de la seva comunitat, comparat amb els formats analògics. Amb els resultats, aquesta tesi aspira a convertir-se en una font de referències que enriqueixi l'escassa bibliografia de la temàtica, proporcionant una llista de formes organitzatives i de comissariat, així com les pràctiques més primerenques mai publicades, i en conjunt contribuir a la historiografia actual sobre dones. Finalment, alhora que es comenta la situació actual de les dones en les àrees d'art i tecnologia, aquest treball presenta Atenea, un projecte de mentorització i networking per a dones en les arts i carreres STEM. Atenea està compost d'un programmea d'activitats amb diferents activitats organitzatives i de comissariat, així com una plataforma de networking. Addicionalment, el projecte té un programmea enfocat a transmetre coneixements STEM a nenes a través de pràctiques artístiques amb tecnologia de la mà de dones artistes nacionals com mentores.[EN] This dissertation explores a series of all-women self-organised initiatives that have emerged at the intersection of art and technology internationally. These grassroot initiatives have continued to appear in places where there is a new media arts scene, generally male-dominated and low in diversity. This research work is a mapping of this techno-social and artistic phenomenon, attempting to identify its earliest practices. Through interviews, surveys and ethnographic research, this thesis seeks to comprehend the context that has given rise to such initiatives over three decades and half, examining the different typologies by formats, extension, space and structure that they have adopted to address the situation of women in these fields. There is a void of references on all-women curatorial and organisational forms in art and technology. Moreover, general literature points at digital subcultures of the 1990s such as cyberfeminism and DIY movement as first referents for women's alliance and translocal activism in the digital age. However, collaboration and segregation strategies in new media arts have not been discussed in feminist scholarship from a deeper historical perspective. Therefore, this dissertation elaborates the linkage of such initiatives to early history of all-women art groups and alternative art spaces, history of women in new media arts and to network cultures. New information and communication technologies have enabled new forms of communication and connectivity cross-borders; however, this work investigates in which measure these initiatives are dependent upon ICT for the establishment of a community, compared to analog formats. With the results, this dissertation aims to become a source of references that enriches the scarce bibliography on the topic, providing a list of curatorial and organisational forms with the earliest and unpublished practices that altogether contribute to current women's historiography. Lastly, while discussing the current situation of women in art and technology fields, this works presents Atenea, a mentoring and networking platform for women in Arts and STEM careers. Atenea is comprised of a programme having different organizational, curatorial and networking activities. Additionally, the platform has a project focused on teaching STEM skills to girls through new media arts with female artists as mentors. Keywords: All-women, self-organised, initiatives, new media arts, digital arts, electronic arts, art, technology, activism, mapping, alternative art spaces, collaboration, segregation, ICT, digital age, virtual, community, network cultures, feminism, cyberfeminism, Atenea, STEM, STEAM, girls, organisational, curatorial, networking platform, mentoring,Robles Mateo, E. (2020). All-Women Initiatives in Art and Technology 1986-2020. Atenea: Mentoring and Networking Project in Steam [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/159248TESI

Animating Unpredictable Effects

Uncanny computer-generated animations of splashing waves, billowing smoke clouds, and characters’ flowing hair have become a ubiquitous presence on screens of all types since the 1980s. This Open Access book charts the history of these digital moving images and the software tools that make them. Unpredictable Visual Effects uncovers an institutional and industrial history that saw media industries conducting more private R&D as Cold War federal funding began to wane in the late 1980s. In this context studios and media software companies took concepts used for studying and managing unpredictable systems like markets, weather, and fluids and turned them into tools for animation. Unpredictable Visual Effects theorizes how these animations are part of a paradigm of control evident across society, while at the same time exploring what they can teach us about the relationship between making and knowing

Web Cube: A New Model for 3-D Web Browsing Based on Hand Gesture Interaction

3-D web browsing is a promising trend for interaction with web content. However it is still illusive between virtual reality applications on the one side, and conventional web browsing on the other. In this research we propose a new model for 3-D web browsing that capitalizes on features of virtual reality technology with those of conventional browsing in order to provide an enhanced interactive user experience with web content. The new model is based on representing information content elements in 3-D perspective and organizing them inside a 3-D container that we call a “Web Cube” for 3-D web browsing. Furthermore, the model defines appropriate interaction mechanisms based on hand gestures. The model has been evaluated using an experimental technique to evaluate its efficiency, and a questionnaire to evaluate user satisfaction

Applied Visualization in the Neurosciences and the Enhancement of Visualization through Computer Graphics

The complexity and size of measured and simulated data in many fields of science is increasing constantly. The technical evolution allows for capturing smaller features and more complex structures in the data. To make this data accessible by the scientists, efficient and specialized visualization techniques are required. Maximum efficiency and value for the user can only be achieved by adapting visualization to the specific application area and the specific requirements of the scientific field. Part I: In the first part of my work, I address the visualization in the neurosciences. The neuroscience tries to understand the human brain; beginning at its smallest parts, up to its global infrastructure. To achieve this ambitious goal, the neuroscience uses a combination of three-dimensional data from a myriad of sources, like MRI, CT, or functional MRI. To handle this diversity of different data types and sources, the neuroscience need specialized and well evaluated visualization techniques. As a start, I will introduce an extensive software called \"OpenWalnut\". It forms the common base for developing and using visualization techniques with our neuroscientific collaborators. Using OpenWalnut, standard and novel visualization approaches are available to the neuroscientific researchers too. Afterwards, I am introducing a very specialized method to illustrate the causal relation of brain areas, which was, prior to that, only representable via abstract graph models. I will finalize the first part of my work with an evaluation of several standard visualization techniques in the context of simulated electrical fields in the brain. The goal of this evaluation was clarify the advantages and disadvantages of the used visualization techniques to the neuroscientific community. We exemplified these, using clinically relevant scenarios. Part II: Besides the data preprocessing, which plays a tremendous role in visualization, the final graphical representation of the data is essential to understand structure and features in the data. The graphical representation of data can be seen as the interface between the data and the human mind. The second part of my work is focused on the improvement of structural and spatial perception of visualization -- the improvement of the interface. Unfortunately, visual improvements using computer graphics methods of the computer game industry is often seen sceptically. In the second part, I will show that such methods can be applied to existing visualization techniques to improve spatiality and to emphasize structural details in the data. I will use a computer graphics paradigm called \"screen space rendering\". Its advantage, amongst others, is its seamless applicability to nearly every visualization technique. I will start with two methods that improve the perception of mesh-like structures on arbitrary surfaces. Those mesh structures represent second-order tensors and are generated by a method named \"TensorMesh\". Afterwards I show a novel approach to optimally shade line and point data renderings. With this technique it is possible for the first time to emphasize local details and global, spatial relations in dense line and point data.In vielen Bereichen der Wissenschaft nimmt die Größe und Komplexität von gemessenen und simulierten Daten zu. Die technische Entwicklung erlaubt das Erfassen immer kleinerer Strukturen und komplexerer Sachverhalte. Um solche Daten dem Menschen zugänglich zu machen, benötigt man effiziente und spezialisierte Visualisierungswerkzeuge. Nur die Anpassung der Visualisierung auf ein Anwendungsgebiet und dessen Anforderungen erlaubt maximale Effizienz und Nutzen für den Anwender. Teil I: Im ersten Teil meiner Arbeit befasse ich mich mit der Visualisierung im Bereich der Neurowissenschaften. Ihr Ziel ist es, das menschliche Gehirn zu begreifen; von seinen kleinsten Teilen bis hin zu seiner Gesamtstruktur. Um dieses ehrgeizige Ziel zu erreichen nutzt die Neurowissenschaft vor allem kombinierte, dreidimensionale Daten aus vielzähligen Quellen, wie MRT, CT oder funktionalem MRT. Um mit dieser Vielfalt umgehen zu können, benötigt man in der Neurowissenschaft vor allem spezialisierte und evaluierte Visualisierungsmethoden. Zunächst stelle ich ein umfangreiches Softwareprojekt namens \"OpenWalnut\" vor. Es bildet die gemeinsame Basis für die Entwicklung und Nutzung von Visualisierungstechniken mit unseren neurowissenschaftlichen Kollaborationspartnern. Auf dieser Basis sind klassische und neu entwickelte Visualisierungen auch für Neurowissenschaftler zugänglich. Anschließend stelle ich ein spezialisiertes Visualisierungsverfahren vor, welches es ermöglicht, den kausalen Zusammenhang zwischen Gehirnarealen zu illustrieren. Das war vorher nur durch abstrakte Graphenmodelle möglich. Den ersten Teil der Arbeit schließe ich mit einer Evaluation verschiedener Standardmethoden unter dem Blickwinkel simulierter elektrischer Felder im Gehirn ab. Das Ziel dieser Evaluation war es, der neurowissenschaftlichen Gemeinde die Vor- und Nachteile bestimmter Techniken zu verdeutlichen und anhand klinisch relevanter Fälle zu erläutern. Teil II: Neben der eigentlichen Datenvorverarbeitung, welche in der Visualisierung eine enorme Rolle spielt, ist die grafische Darstellung essenziell für das Verständnis der Strukturen und Bestandteile in den Daten. Die grafische Repräsentation von Daten bildet die Schnittstelle zum Gehirn des Menschen. Der zweite Teile meiner Arbeit befasst sich mit der Verbesserung der strukturellen und räumlichen Wahrnehmung in Visualisierungsverfahren -- mit der Verbesserung der Schnittstelle. Leider werden viele visuelle Verbesserungen durch Computergrafikmethoden der Spieleindustrie mit Argwohn beäugt. Im zweiten Teil meiner Arbeit werde ich zeigen, dass solche Methoden in der Visualisierung angewendet werden können um den räumlichen Eindruck zu verbessern und Strukturen in den Daten hervorzuheben. Dazu nutze ich ein in der Computergrafik bekanntes Paradigma: das \"Screen Space Rendering\". Dieses Paradigma hat den Vorteil, dass es auf nahezu jede existierende Visualiserungsmethode als Nachbearbeitunsgschritt angewendet werden kann. Zunächst führe ich zwei Methoden ein, die die Wahrnehmung von gitterartigen Strukturen auf beliebigen Oberflächen verbessern. Diese Gitter repräsentieren die Struktur von Tensoren zweiter Ordnung und wurden durch eine Methode namens \"TensorMesh\" erzeugt. Anschließend zeige ich eine neuartige Technik für die optimale Schattierung von Linien und Punktdaten. Mit dieser Technik ist es erstmals möglich sowohl lokale Details als auch globale räumliche Zusammenhänge in dichten Linien- und Punktdaten zu erfassen