2,791 research outputs found
4DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacements
This work presents 4DHumanOutfit, a new dataset of densely sampled
spatio-temporal 4D human motion data of different actors, outfits and motions.
The dataset is designed to contain different actors wearing different outfits
while performing different motions in each outfit. In this way, the dataset can
be seen as a cube of data containing 4D motion sequences along 3 axes with
identity, outfit and motion. This rich dataset has numerous potential
applications for the processing and creation of digital humans, e.g. augmented
reality, avatar creation and virtual try on. 4DHumanOutfit is released for
research purposes at https://kinovis.inria.fr/4dhumanoutfit/. In addition to
image data and 4D reconstructions, the dataset includes reference solutions for
each axis. We present independent baselines along each axis that demonstrate
the value of these reference solutions for evaluation tasks
RECREATING AND SIMULATING DIGITAL COSTUMES FROM A STAGE PRODUCTION OF \u3ci\u3eMEDEA\u3c/i\u3e
This thesis investigates a technique to effectively construct and simulate costumes from a stage production Medea, in a dynamic cloth simulation application like Maya\u27s nDynamics. This was done by using data collected from real-world fabric tests and costume construction in the theatre\u27s costume studio. Fabric tests were conducted and recorded, by testing costume fabrics for drape and behavior with two collision objects. These tests were recreated digitally in Maya to derive appropriate parameters for the digital fabric, by comparing with the original reference. Basic mannequin models were created using the actors\u27 measurements and skeleton-rigged to enable animation. The costumes were then modeled and constrained according to the construction process observed in the costume studio to achieve the same style and stitch as the real costumes. Scenes selected and recorded from Medea were used as reference to animate the actors\u27 models. The costumes were assigned the parameters derived from the fabric tests to produce the simulations. Finally, the scenes were lit and rendered out to obtain the final videos which were compared to the original recordings to ascertain the accuracy of simulation. By obtaining and refining simulation parameters from simple fabric collision tests, and modeling the digital costumes following the procedures derived from real-life costume construction, realistic costume simulation was achieved
PERGAMO: Personalized 3D Garments from Monocular Video
Clothing plays a fundamental role in digital humans. Current approaches to
animate 3D garments are mostly based on realistic physics simulation, however,
they typically suffer from two main issues: high computational run-time cost,
which hinders their development; and simulation-to-real gap, which impedes the
synthesis of specific real-world cloth samples. To circumvent both issues we
propose PERGAMO, a data-driven approach to learn a deformable model for 3D
garments from monocular images. To this end, we first introduce a novel method
to reconstruct the 3D geometry of garments from a single image, and use it to
build a dataset of clothing from monocular videos. We use these 3D
reconstructions to train a regression model that accurately predicts how the
garment deforms as a function of the underlying body pose. We show that our
method is capable of producing garment animations that match the real-world
behaviour, and generalizes to unseen body motions extracted from motion capture
dataset.Comment: Published at Computer Graphics Forum (Proc. of ACM/SIGGRAPH SCA),
2022. Project website http://mslab.es/projects/PERGAMO
Tracking and Retexturing Cloth for RealTime Virtual Clothing Applications
Abstract. In this paper, we describe a dynamic texture overlay method from monocular images for real-time visualization of garments in a virtual mirror environment. Similar to looking into a mirror when trying on clothes, we create the same impression but for virtually textured garments. The mirror is replaced by a large display that shows the mirrored image of a camera capturing e.g. the upper body part of a person. By estimating the elastic deformations of the cloth from a single camera in the 2D image plane and recovering the illumination of the textured surface of a shirt in real time, an arbitrary virtual texture can be realistically augmented onto the moving garment such that the person seems to wear the virtual clothing. The result is a combination of the real video and the new augmented model yielding a realistic impression of the virtual piece of cloth
Practical Color-Based Motion Capture
Motion capture systems have been widely used for high quality content creation and virtual reality but are rarely used in consumer applications due to their price and setup cost. In this paper, we propose a motion capture system built from commodity components that can be deployed in a matter of minutes. Our approach uses one or more webcams and a color shirt to track the upper-body at interactive rates. We describe a robust color calibration system that enables our color-based tracking to work against cluttered backgrounds and under multiple illuminants. We demonstrate our system in several real-world indoor and outdoor settings
Learning to Dress {3D} People in Generative Clothing
Three-dimensional human body models are widely used in the analysis of human
pose and motion. Existing models, however, are learned from minimally-clothed
3D scans and thus do not generalize to the complexity of dressed people in
common images and videos. Additionally, current models lack the expressive
power needed to represent the complex non-linear geometry of pose-dependent
clothing shapes. To address this, we learn a generative 3D mesh model of
clothed people from 3D scans with varying pose and clothing. Specifically, we
train a conditional Mesh-VAE-GAN to learn the clothing deformation from the
SMPL body model, making clothing an additional term in SMPL. Our model is
conditioned on both pose and clothing type, giving the ability to draw samples
of clothing to dress different body shapes in a variety of styles and poses. To
preserve wrinkle detail, our Mesh-VAE-GAN extends patchwise discriminators to
3D meshes. Our model, named CAPE, represents global shape and fine local
structure, effectively extending the SMPL body model to clothing. To our
knowledge, this is the first generative model that directly dresses 3D human
body meshes and generalizes to different poses. The model, code and data are
available for research purposes at https://cape.is.tue.mpg.de.Comment: CVPR-2020 camera ready. Code and data are available at
https://cape.is.tue.mpg.d
Recommended from our members
A video-based automated recommender (VAR) system for garments
In this paper, we propose an automated and scalable garment recommender system using real-time in-store videos that can improve the experiences of garment shoppers and increase product sales. The video-based automated recommender (VAR) system is based on observations that garment shoppers tend to try on garments and evaluate themselves in front of store mirrors. Combining state-of-the-art computer vision techniques with marketing models of consumer preferences, the system automatically identifies shoppersâ preferences based on their reactions and uses that information to make meaningful personalized recommendations. First, the system uses a camera to capture a shopperâs behavior in front of the mirror to make inferences about her preferences based on her facial expressions and the part of the garment she is examining at each time point. Second, the system identifies shoppers with preferences similar to the focal customer from a database of shoppers whose preferences, purchasing, and/or consideration decisions are known. Finally, recommendations are made to the focal customer based on the preferences, purchasing, and/or consideration decisions of these like-minded shoppers. Each of the three steps can be implemented with several variations, and a retailing chain can choose the specific configuration that best serves its purpose. In this paper, we present an empirical test that compares one specific type of VAR system implementation against two alternative, nonautomated personal recommender systems: self-explicated conjoint (SEC) and self-evaluation after try-on (SET). The results show that VAR consistently outperforms SEC and SET. A second empirical study demonstrates the feasibility of VAR in real-time applications. Participants in the second study enjoyed the VAR experience, and almost all of them tried on the recommended garments. VAR should prove to be a valuable tool for both garment retailers and shoppers.
Data, as supplemental material, are available at http://dx.doi.org/10.1287/mksc.2016.0984.The authors thank the participants in presentations given by the authors in College of Business at City Univeristy of HongKong and Cambridge Judge Business School for their feedback, as well as the Editor, the Area Editor, and two anonymous Marketing Science reviewers for their insightful comments. This research was supported by two National Natural Science Foundation of China Fund (Grants 71232008 & 71502039), and the Institute for Sustainable Innovation and Growth (iSIG) at School of Management, Fudan University
New editing techniques for video post-processing
This thesis contributes to capturing 3D cloth shape, editing cloth texture and altering object shape and motion in multi-camera and monocular video recordings. We propose a technique to capture cloth shape from a 3D scene flow by determining optical flow in several camera views. Together with a silhouette matching constraint we can track and reconstruct cloth surfaces in long video sequences. In the area of garment motion capture, we present a system to reconstruct time-coherent triangle meshes from multi-view video recordings. Texture mapping of the acquired triangle meshes is used to replace the recorded texture with new cloth patterns. We extend this work to the more challenging single camera view case. Extracting texture deformation and shading effects simultaneously enables us to achieve texture replacement effects for garments in monocular video recordings. Finally, we propose a system for the keyframe editing of video objects. A color-based segmentation algorithm together with automatic video inpainting for filling in missing background texture allows us to edit the shape and motion of 2D video objects. We present examples for altering object trajectories, applying non-rigid deformation and simulating camera motion.In dieser Dissertation stellen wir BeitrĂ€ge zur 3D-Rekonstruktion von StoffoberfĂ€chen, zum Editieren von Stofftexturen und zum Editieren von Form und Bewegung von Videoobjekten in Multikamera- und Einkamera-Aufnahmen vor. Wir beschreiben eine Methode fĂŒr die 3D-Rekonstruktion von StoffoberflĂ€chen, die auf der Bestimmung des optischen FluĂ in mehreren Kameraansichten basiert. In Kombination mit einem Abgleich der Objektsilhouetten im Video und in der Rekonstruktion erhalten wir Rekonstruktionsergebnisse fĂŒr lĂ€ngere Videosequenzen. FĂŒr die Rekonstruktion von KleidungsstĂŒcken beschreiben wir ein System, das zeitlich kohĂ€rente Dreiecksnetze aus Multikamera-Aufnahmen rekonstruiert. Mittels Texturemapping der erhaltenen Dreiecksnetze wird die Stofftextur in der Aufnahme mit neuen Texturen ersetzt. Wir setzen diese Arbeit fort, indem wir den anspruchsvolleren Fall mit nur einer einzelnen Videokamera betrachten. Um realistische Resultate beim Ersetzen der Textur zu erzielen, werden sowohl Texturdeformationen durch zugrundeliegende Deformation der OberflĂ€che als auch Beleuchtungseffekte berĂŒcksichtigt. Im letzten Teil der Dissertation stellen wir ein System zum Editieren von Videoobjekten mittels Keyframes vor. Dies wird durch eine Kombination eines farbbasierten Segmentierungsalgorithmus mit automatischem AuffĂŒllen des Hintergrunds erreicht, wodurch Form und Bewegung von 2D-Videoobjekten editiert werden können. Wir zeigen Beispiele fĂŒr editierte Objekttrajektorien, beliebige Deformationen und simulierte Kamerabewegung
Developing shape change-based fashion prototyping strategies:Enhancing computational thinking in fashion practice and creativity
Emerging technologies enable fluid and versatile material forms of fashionable wearables and e-textiles, with experts in engineering and material science proposing numerous strategies for dynamic textile and garment structures to satisfy various needs. Nevertheless, a critical gap remains in developing practical fashion prototyping strategies that fuse with computational thinking to challenge current norms and envision the future of fashion. This study introduces shape change-based fashion prototyping as a design strategy for dynamic expressions and affordances to inspire fashion practitionersâ interdisciplinary endeavors. We present three studio-based practices as case studies to demonstrate how shape-changing mechanisms including servo motors, shape memory alloys, and pneumatics, spur new fashion construction skills and broaden the scope of potential applications. By doing so, this study contributes to material and conceptual innovation, creating pathways for the seamless integration of technologies from conceptualization, and implementation to envision. Our findings shed light on design possibilities and challenges and offer design recommendations that guide future endeavors. The implications of our research underscore the importance of adopting a relational approach to design variables, emphasize the value of fostering shared vocabulary between fashion and technical design, and highlight the transformative potential of shape-changing prototyping in reshaping the intricate body-material relationship.</p
Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation
Authoring dynamic garment shapes for character animation on body motion is one of the fundamental steps in the CG industry. Established workflows are either time and labor consuming (i.e., manual editing on dense frames with controllers), or lack keyframe-level control (i.e., physically-based simulation). Not surprisingly, garment authoring remains a bottleneck in many production pipelines. Instead, we present a deep-learning-based approach for semi-automatic authoring of garment animation, wherein the user provides the desired garment shape in a selection of keyframes, while our system infers a latent representation for its motion-independent intrinsic parameters (e.g., gravity, cloth materials, etc.). Given new character motions, the latent representation allows to automatically generate a plausible garment animation at interactive rates. Having factored out character motion, the learned intrinsic garment space enables smooth transition between keyframes on a new motion sequence. Technically, we learn an intrinsic garment space with an motion-driven autoencoder network, where the encoder maps the garment shapes to the intrinsic space under the condition of body motions, while the decoder acts as a differentiable simulator to generate garment shapes according to changes in character body motion and intrinsic parameters. We evaluate our approach qualitatively and quantitatively on common garment types. Experiments demonstrate our system can significantly improve current garment authoring workflows via an interactive user interface. Compared with the standard CG pipeline, our system significantly reduces the ratio of required keyframes from 20% to 1 -- 2%
- âŠ