Free-form Shape Modeling in XR: A Systematic Review

Shape modeling research in Computer Graphics has been an active area for decades. The ability to create and edit complex 3D shapes has been of key importance in Computer-Aided Design, Animation, Architecture, and Entertainment. With the growing popularity of Virtual and Augmented Reality, new applications and tools have been developed for artistic content creation; real-time interactive shape modeling has become increasingly important for a continuum of virtual and augmented reality environments (eXtended Reality (XR)). Shape modeling in XR opens new possibilities for intuitive design and shape modeling in an accessible way. Artificial Intelligence (AI) approaches generating shape information from text prompts are set to change how artists create and edit 3D models. There has been a substantial body of research on interactive 3D shape modeling. However, there is no recent extensive review of the existing techniques and what AI shape generation means for shape modeling in interactive XR environments. In this state-of-the-art paper, we fill this research gap in the literature by surveying free-form shape modeling work in XR, with a focus on sculpting and 3D sketching, the most intuitive forms of free-form shape modeling. We classify and discuss these works across five dimensions: contribution of the articles, domain setting, interaction tool, auto-completion, and collaborative designing. The paper concludes by discussing the disconnect between interactive 3D sculpting and sketching and how this will likely evolve with the prevalence of AI shape-generation tools in the future

HairBrush for Immersive Data-Driven Hair Modeling

International audienceWhile hair is an essential component of virtual humans, it is also one of the most challenging digital assets to create. Existing automatic techniques lack the generality and flexibility to create rich hair variations, while manual authoring interfaces often require considerable artistic skills and efforts, especially for intricate 3D hair structures that can be difficult to navigate. We propose an interactive hair modeling system that can help create complex hairstyles in minutes or hours that would otherwise take much longer with existing tools. Modelers, including novice users, can focus on the overall hairstyles and local hair deformations, as our system intelligently suggests the desired hair parts. Our method combines the flexibility of manual authoring and the convenience of data-driven automation. Since hair contains intricate 3D structures such as buns, knots, and strands, they are inherently challenging to create using traditional 2D interfaces. Our system provides a new 3D hair author-ing interface for immersive interaction in virtual reality (VR). Users can draw high-level guide strips, from which our system predicts the most plausible hairstyles via a deep neural network trained from a professionally curated dataset. Each hairstyle in our dataset is composed of multiple variations, serving as blend-shapes to fit the user drawings via global blending and local deformation. The fitted hair models are visualized as interactive suggestions that the user can select, modify, or ignore. We conducted a user study to confirm that our system can significantly reduce manual labor while improve the output quality for modeling a variety of head and facial hairstyles that are challenging to create via existing techniques

DupRobo: An Interactive Robotic Platform for Physical Block-Based Autocompletion

In this paper, we present DupRobo, an interactive robotic platform for tangible block-based design and construction. DupRobo supported user-customisable exemplar, repetition control, and tangible autocompletion, through the computer-vision and the robotic techniques. With DupRobo, we aim to reduce users’ workload in repetitive block-based construction, yet preserve the direct manipulatability and the intuitiveness in tangible model design, such as product design and architecture design

Autocomplete element fields and interactive synthesis system development for aggregate applications.

Aggregate elements are ubiquitous in natural and man-made objects and have played an important role in the application of graphics, design and visualization. However, to efficiently arrange these aggregate elements with varying anisotropy and deformability still remains challenging, in particular in 3D environments. To overcome such a thorny issue, we thus introduce autocomplete element fields, including an element distribution formulation that can effectively cope with diverse output compositions with controllable element distributions in high production standard and efficiency as well as an element field formulation that can smoothly orient all the synthesized elements following given inputs, such as scalar or direction fields. The pro- posed formulations can not only properly synthesize distinct types of aggregate elements across various domain spaces without incorporating any extra process but also directly compute complete element fields from partial specifications without requiring fully specified inputs in any algorithmic step. Furthermore, in order to reduce input workload and enhance output quality for better usability and interactivity, we further develop an interactive synthesis system, centered on the idea of our autocomplete element fields, to facilitate the creation of element aggregations within different output do- mains. Analogous to conventional painting workflows, through a palette- based brushing interface, users can interactively mix and place a few aggregate elements over a brushing canvas and let our system automatically populate more aggregate elements with intended orientations and scales for the rest of outcome. The developed system can empower the users to iteratively design a variety of novel mixtures with reduced workload and enhanced quality under an intuitive and user-friendly brushing workflow with- out the necessity of a great deal of manual labor or technical expertise. We validate our prototype system with a pilot user study and exhibit its application in 2D graphic design, 3D surface collage, and 3D aggregate modeling

The Use of Games and Crowdsourcing for the Fabrication-aware Design of Residential Buildings

State-of-the-art participatory design acknowledges the true, ill-defined nature of design problems, taking into account stakeholders' values and preferences. However, it overburdens the architect, who has to synthesize far more constraints into a one-of-a-kind design. Generative Design promises to equip architects with great power to standardize and systemize the design process. However, the common trap of generative design is trying to treat architecture simply as a tame problem. In this work, I investigate the use of games and crowdsourcing in architecture through two sets of explorative questions. First, if everyone can participate in the network-enabled creation of the built environment, what role will they play? And what tools will they need to enable them? And second, if anyone can use digital fabrication to build any building, how will we design it? What design paradigms will govern this process? I present a map of design paradigms that lie at the intersections of Participatory Design, Generative Design, Game Design, and Crowd Wisdom. In four case studies, I explore techniques to employ the practices from the four fields in the service of architecture. Generative Design can lower the difficulty of the challenge to design by automating a large portion of the work. A newly formulated, unified taxonomy of generative design across the disciplines of architecture, computer science, and computer games builds the base for the use of algorithms in the case studies. The work introduces Playable Voxel-Shape Grammars, a new type of generative technique. It enables Game Design to guide participants through a series of challenges, effectively increasing their skills by helping them understand the underlying principles of the design task at hand. The use of crowdsourcing in architecture can mean thousands of architects creating content for a generative design system, to expand and open up its design space. Crowdsourcing can also be about millions of people online creating designs that an architect or a homeowner can refer to increase their understanding of the complex issues at hand in a given design project and for better decision making. At the same time, game design in architecture helps find the balance between algorithmically exploring pre-defined design alternatives and open-ended, free creativity. The research reveals a layered structure of entry points for crowd-contributed content as well as the granular nature of authorship among four different roles: non-expert stakeholders, architects, the crowd, and the tool-makers