Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Design, creation and implementation of characters for video games Final Degree Work report

Treball final de Grau en Disseny i Desenvolupament de Videojocs. Codi: VJ1241. Curs acadèmic: 2018/2019This document has been created in order to explain the process of character modeling for video games and its implementation in a game engine. Its main objective is that the reader can understand, without going into technical details, the general process followed to develop a character of these characteristics. It is interesting because many people do not know the effort and work it takes to make the video games they consume

How Will It Drape Like? Capturing Fabric Mechanics from Depth Images

We propose a method to estimate the mechanical parameters of fabrics using a casual capture setup with a depth camera. Our approach enables to create mechanically-correct digital representations of real-world textile materials, which is a fundamental step for many interactive design and engineering applications. As opposed to existing capture methods, which typically require expensive setups, video sequences, or manual intervention, our solution can capture at scale, is agnostic to the optical appearance of the textile, and facilitates fabric arrangement by non-expert operators. To this end, we propose a sim-to-real strategy to train a learning-based framework that can take as input one or multiple images and outputs a full set of mechanical parameters. Thanks to carefully designed data augmentation and transfer learning protocols, our solution generalizes to real images despite being trained only on synthetic data, hence successfully closing the sim-to-real loop.Key in our work is to demonstrate that evaluating the regression accuracy based on the similarity at parameter space leads to an inaccurate distances that do not match the human perception. To overcome this, we propose a novel metric for fabric drape similarity that operates on the image domain instead on the parameter space, allowing us to evaluate our estimation within the context of a similarity rank. We show that out metric correlates with human judgments about the perception of drape similarity, and that our model predictions produce perceptually accurate results compared to the ground truth parameters.Comment: 12 pages, 12 figures. Accepted to EUROGRAPHICS 2023. Project website: https://carlosrodriguezpardo.es/projects/MechFromDepth

Videopelit ja pukutaide - analogisten pukusuunnittelumetodien digitalisointi

This thesis explores ways of integrating a costume professional to the character art team in the game industry. The research suggests, that integrating costume knowledge into the character design pipeline increases the storytelling value of the characters and provides tools for the narrative. The exploration of integrating a costume professional into game character creation as a process is still rare and little information of costume in games and experiences in transferring an analogue character building skillset into a digital one can be found, therefore this research was generated to provide knowledge on the subject. The research's main emphasis is on immersion-driven AAA-games that employ 3D-graphics and human characters and are either photorealistic or represent stylized realism. Technology for depicting reality is advancing and digital industries have become aware of the extensive skills required to depict increasingly realistic worlds. Also, tools for character art are beginning to lean on actual costume construction: the pattern based cloth simulation software entitled Marvelous Designer has become the industry standard for character clothing. The material of this thesis is based on the author's experience as an intern and Costume Artist at the game company Remedy Entertainment and on data collection in the form of participant observation, conversational interviews, archival searches and assorted documents as an internal employee of the company. Therefore, an ethnographical research that applies to qualitative, descriptive, nonmathematical and naturalistic research methods is utilized in this thesis. The result of this research is a costume production pipeline for integrating a costume professional into the game character design process. It is formed by comparing costuming processes of game and film industries to explore the similarities and differences in methods to analyze the most effective combination of these two. The final pipeline introduces the costume professional’s position during the different stages of the character design process. Furthermore, the thesis categorizes aspects essential for a costume designer to internalize in order to become a functional part of the Character Art team and the skills and knowledge required to support the character design in a production. This research identifies the need for costume knowledge in realistic AAA-games. When employing a costume professional into a game production, this thesis offers tools and vocabulary for collaboration. Costume designers are Character Artists, but with different tools and skill set and costume design can be seen as a live form of character art.Tämä opinnäytetyö käsittelee pukusuunnittelua peliteollisuudessa ja pukusuunnittelijan tarvetta hahmosuunnitteluprosessissa. Tutkimuksen materiaali perustuu kirjoittajan omaan kokemukseen harjoittelijana ja pukusuunnittelijana (Costume Artist) Remedy Entertainment -peliyhtiön hahmosuunnittelutiimissä. Työ esittelee eri peliyhtiöiden hahmotiimien käyttämiä pukusuunnittelumetodeja ja arvioi, miten pukusuunnittelun tiedostaminen erillisenä osana hahmosuunnitteluprosessia ja puvun kerronnallisten elementtien tunnistaminen ja hyödyntäminen tukee uskottavan pelihahmon luomista realistisissa peleissä. Tutkimuksen lähtökohtana on ajatus, että pukusuunnittelun integroiminen hahmosuunnitteluprosessiin syventää hahmoa ja kuluttajan pelikokemusta ja näin ollen nostaa pelin arvoa tuotteena. Tutkimuksen tuloksena syntyi uusi pukusuunnittelun tuotantolinja (Costume production pipeline) peliteollisuuden käyttöön. Se esittelee pukusuunnittelijan asemoitumisen pelituotannon hahmosuunnitteluprosessiin. Teknologinen kehitys mahdollistaa yhä realistisempien digitaalisten todellisuuksien luomisen, jonka seurauksena peliteollisuus työllistää enenevässä määrin myös perinteisten alojen erityisosaajia, kuten arkkitehtejä sekä elokuva-alan valosuunnittelijoita. Myös digitaalisen hahmonluonnin työvälineet nojaavat jo tosielämän vaatetuotantoon: vaatesimulaatio-ohjelma Marvelous Designer perustuu vaatteiden kaavoihin ja on laajalti käytössä pelihahmojen suunnittelussa. Pelit, joita tutkimus tarkastelee, ovat AAA-pelejä, jotka hyödyntävät 3D-grafiikkaa ja ihmishahmoja ja jotka ovat joko photorealistisia tai edustavat tyyliteltyä realismia. Tutkimuksen materiaali on koostettu keskustelullisista, strukturoimattomista haastatteluista, kirjoittajan työpäiväkirjasta, yhtiön edellisten pelien arkistomateriaaleista, sekä yhtiön sisäisistä ohjeistuksista. Tutkimus esittelee keinoja, joilla fyysisen ihmisvartalon kanssa työskentelemään kouluttautunut pukusuunnittelija voi integroitua digitaaliseen hahmonluontiprosessiin ja osoittaa, että pukusuunnittelijat ovat analogisilla metodeilla työskenteleviä hahmosuunnittelijoita. Hahmosuunnittelun etu suhteessa esitystaiteen pukusuunnitteluun on vapaus näyttelijän fyysisen ruumiin sekä fysiikan lakien tuottamista rajoitteista. Vaatesuunnittelu keskittyy vaatteeseen ja pukusuunnittelu fyysisen ihmisruumiin ja vaatteen synnyttämään kombinaatioon ja interaktioon, mutta hahmosuunnittelu tarjoaa pukusuunnittelullisesti rajoittamattoman vapauden hahmotulkintaan