HandPainter – 3D sketching in VR with hand-based physical proxy

3D sketching in virtual reality (VR) enables users to create 3D virtual objects intuitively and immersively. However, previous studies showed that mid-air drawing may lead to inaccurate sketches. To address this issue, we propose to use one hand as a canvas proxy and the index finger of the other hand as a 3D pen. To this end, we first perform a formative study to compare two-handed interaction with tablet-pen interaction for VR sketching. Based on the findings of this study, we design HandPainter, a VR sketching system which focuses on the direct use of two hands for 3D sketching without requesting any tablet, pen, or VR controller. Our implementation is based on a pair of VR gloves, which provide hand tracking and gesture capture. We devise a set of intuitive gestures to control various functionalities required during 3D sketching, such as canvas panning and drawing positioning. We show the effectiveness of HandPainter by presenting a number of sketching results and discussing the outcomes of a user study-based comparison with mid-air drawing and tablet-based sketching tools

Development of design tools for the evaluation of complex CAD models

The goal of this thesis is to invent means to enable 3D-models, created in SOLIDWORKS CAD-software, to be explored in a low-cost head-mounted display device Oculus Rift. Such a design tool would make possible to study 3D CAD-models in Virtual Reality with a very low cost and share design information for people who are not so familiar with CAD-models. As a result this can lower the product development effort for new products enabling, lower working time and improve both the quality of the final product and customer satisfaction. CAD-models as well as exported graphics format from SOLIDWORKS can’t directly be imported to Oculus Rift device. Therefore transformation pipeline need to be invented between the SOLIDWORKS and Oculus Rift. There are some options for this and their issues are studied in this thesis. As a best option it was found to use a conversion plugin tool to SOLIDWORKS which convert CAD-models to FBX-format. This format can be imported to Unity game engine software which is supported by Oculus Rift device. Graphics file conversion process in known to be imperfect. Therefore conversion of three SOLIDWORKS CAD-models have been analyzed; a Crane, a Pipe system, and a Hydraulic cylinder. This testing gave a good overview of possibilities and limitations of the graphics pipeline. It was found that ordinary CAD-models can be transferred to Oculus Rift without problems. However, there are some special features in CAD models which cause problems for in graphics file conversion

Conceptual free-form styling in virtual environments

This dissertation introduces the tools for designing complete models from scratch directly in a head-tracked, table-like virtual work environment. The models consist of free-form surfaces, and are constructed by drawing a network of curves directly in space. This is accomplished by using a tracked pen-like input device. Interactive deformation tools for curves and surfaces are proposed and are based on variational methods. By aligning the model with the left hand, editing is made possible with the right hand, corresponding to a natural distribution of tasks using both hands. Furthermore, in the emerging field of 3D interaction in virtual environments, particularly with regard to system control, this work uses novel methods to integrate system control tasks, such as selecting tools, and workflow of shape design. The aim of this work is to propose more suitable user interfaces to computersupported conceptual shape design applications. This would be beneficial since it is a field that lacks adequate support from standard desktop systems.Diese Dissertation beschreibtWerkzeuge zum Entwurf kompletter virtueller Modelle von Grund auf. Dies geschieht direkt in einer tischartigen, virtuellen Arbeitsumge-bung mit Hilfe von Tracking der Hände und der Kopfposition. Die Modelle sind aus Freiformlächen aufgebaut und werden als Netz von Kurven mit Hilfe eines getrack-ten, stiftartigen Eingabegerätes direkt im Raum gezeichnet. Es werden interaktive Deformationswerkzeuge für Kurven und Flächen vorgestellt, die auf Methoden des Variational Modeling basieren. Durch das Ausrichten des Modells mit der linken Hand wird das Editieren mit der rechten Hand erleichtert. Dies entspricht einer natürlichen Aufteilung von Aufgaben auf beide Hände. Zusätzlich stellt diese Arbeit neue Techniken für die 3D-Interaktion in virtuellen Umgebungen, insbesondere im Bereich Anwendungskontrolle, vor, die die Aufgabe der Werkzeugauswahl in den Arbeitsablauf der Formgestaltung integrieren. Das Ziel dieser Arbeit ist es, besser geeignete Schnittstellen für den computer-unterstützten, konzeptionellen Formentwurf zur Verfügung zu stellen; ein Gebiet, für das Standard-Desktop-Systeme wenig geeignete Unterstützung bieten

Conceptual free-form styling in virtual environments

This dissertation introduces the tools for designing complete models from scratch directly in a head-tracked, table-like virtual work environment. The models consist of free-form surfaces, and are constructed by drawing a network of curves directly in space. This is accomplished by using a tracked pen-like input device. Interactive deformation tools for curves and surfaces are proposed and are based on variational methods. By aligning the model with the left hand, editing is made possible with the right hand, corresponding to a natural distribution of tasks using both hands. Furthermore, in the emerging field of 3D interaction in virtual environments, particularly with regard to system control, this work uses novel methods to integrate system control tasks, such as selecting tools, and workflow of shape design. The aim of this work is to propose more suitable user interfaces to computersupported conceptual shape design applications. This would be beneficial since it is a field that lacks adequate support from standard desktop systems.Diese Dissertation beschreibtWerkzeuge zum Entwurf kompletter virtueller Modelle von Grund auf. Dies geschieht direkt in einer tischartigen, virtuellen Arbeitsumge-bung mit Hilfe von Tracking der Hände und der Kopfposition. Die Modelle sind aus Freiformlächen aufgebaut und werden als Netz von Kurven mit Hilfe eines getrack-ten, stiftartigen Eingabegerätes direkt im Raum gezeichnet. Es werden interaktive Deformationswerkzeuge für Kurven und Flächen vorgestellt, die auf Methoden des Variational Modeling basieren. Durch das Ausrichten des Modells mit der linken Hand wird das Editieren mit der rechten Hand erleichtert. Dies entspricht einer natürlichen Aufteilung von Aufgaben auf beide Hände. Zusätzlich stellt diese Arbeit neue Techniken für die 3D-Interaktion in virtuellen Umgebungen, insbesondere im Bereich Anwendungskontrolle, vor, die die Aufgabe der Werkzeugauswahl in den Arbeitsablauf der Formgestaltung integrieren. Das Ziel dieser Arbeit ist es, besser geeignete Schnittstellen für den computer-unterstützten, konzeptionellen Formentwurf zur Verfügung zu stellen; ein Gebiet, für das Standard-Desktop-Systeme wenig geeignete Unterstützung bieten

Embodied Interactions for Spatial Design Ideation: Symbolic, Geometric, and Tangible Approaches

Computer interfaces are evolving from mere aids for number crunching into active partners in creative processes such as art and design. This is, to a great extent, the result of mass availability of new interaction technology such as depth sensing, sensor integration in mobile devices, and increasing computational power. We are now witnessing the emergence of maker culture that can elevate art and design beyond the purview of enterprises and professionals such as trained engineers and artists. Materializing this transformation is not trivial; everyone has ideas but only a select few can bring them to reality. The challenge is the recognition and the subsequent interpretation of human actions into design intent

Collision Detection and Merging of Deformable B-Spline Surfaces in Virtual Reality Environment

This thesis presents a computational framework for representing, manipulating and merging rigid and deformable freeform objects in virtual reality (VR) environment. The core algorithms for collision detection, merging, and physics-based modeling used within this framework assume that all 3D deformable objects are B-spline surfaces. The interactive design tool can be represented as a B-spline surface, an implicit surface or a point, to allow the user a variety of rigid or deformable tools. The collision detection system utilizes the fact that the blending matrices used to discretize the B-spline surface are independent of the position of the control points and, therefore, can be pre-calculated. Complex B-spline surfaces can be generated by merging various B-spline surface patches using the B-spline surface patches merging algorithm presented in this thesis. Finally, the physics-based modeling system uses the mass-spring representation to determine the deformation and the reaction force values provided to the user. This helps to simulate realistic material behaviour of the model and assist the user in validating the design before performing extensive product detailing or finite element analysis using commercially available CAD software. The novelty of the proposed method stems from the pre-calculated blending matrices used to generate the points for graphical rendering, collision detection, merging of B-spline patches, and nodes for the mass spring system. This approach reduces computational time by avoiding the need to solve complex equations for blending functions of B-splines and perform the inversion of large matrices. This alternative approach to the mechanical concept design will also help to do away with the need to build prototypes for conceptualization and preliminary validation of the idea thereby reducing the time and cost of concept design phase and the wastage of resources

Interactions gestuelles multi-point et géométrie déformable pour l’édition 3D sur écran tactile

Despite the advances made in the fields of existing objects capture and of procedural generation, creation of content for virtual worlds can not be perform without human interaction. This thesis suggests to exploit new touch devices ("multi-touch" screens) to obtain an easy, intuitive 2D interaction in order to navigate inside a virtual environment, to manipulate, position and deform 3D objects.First, we study the possibilities and limitations of the hand and finger gestures while interacting on a touch screen in order to discover which gestures are the most adapted to edit 3D scene and environment. In particular, we evaluate the effective number of degrees of freedom of the human hand when constrained on a planar surface. Meanwhile, we develop a new gesture analysis method using phases to identify key motion of the hand and fingers in real time. These results, combined to several specific user-studies, lead to a gestural design pattern which handle not only navigation (camera positioning), but also object positioning, rotation and global scaling. Then, this pattern is extended to complex deformation (such as adding and deleting material, bending or twisting part of objects, using local control). Using these results, we are able to propose and evaluate a 3D world editing interface that handle a naturaltouch interaction, in which mode selection (i.e. navigation, object positioning or object deformation) and task selections is automatically processed by the system, relying on the gesture and the interaction context (without any menu or button). Finally, we extend this interface to integrate more complex deformations, adapting the garment transfer from a character to any other in order to process interactive deformation of the garment while the wearing character is deformed.Malgré les progrès en capture d’objets réels et en génération procédurale, la création de contenus pour les mondes virtuels ne peut se faire sans interaction humaine. Cette thèse propose d’exploiter les nouvelles technologies tactiles (écrans "multi-touch") pour offrir une interaction 2D simple et intuitive afin de naviguer dans un environnement virtuel, et d’y manipuler, positionner et déformer des objets 3D.En premier lieu, nous étudions les possibilité et les limitations gestuelles de la main et des doigts lors d’une interaction sur écran tactile afin de découvrir quels gestes semblent les plus adaptés à l’édition des environnements et des objets 3D. En particulier, nous évaluons le nombre de degré de liberté efficaces d’une main humaine lorsque son geste est contraint à une surface plane. Nous proposons également une nouvelle méthode d’analyse gestuelle par phases permettant d’identifier en temps réel les mouvements clés de la main et des doigts. Ces résultats, combinés à plusieurs études utilisateur spécifiques, débouchent sur l’identification d’un patron pour les interactions gestuelles de base incluant non seulement navigation (placement de caméra), mais aussi placement, rotation et mise à l’échelle des objets. Ce patron est étendudans un second temps aux déformations complexes (ajout et suppression de matière ainsi que courbure ou torsion des objets, avec contrôle de la localité). Tout ceci nous permet de proposer et d’évaluer une interface d’édition des mondes 3D permettant une interaction tactile naturelle, pour laquelle le choix du mode (navigation, positionnement ou déformation) et des tâches correspondantes est automatiquement géré par le système en fonction du geste et de son contexte (sans menu ni boutons). Enfin, nous étendons cette interface pour y intégrer des déformations plus complexe à travers le transfert de vêtements d’un personnage à un autre, qui est étendu pour permettre la déformation interactive du vêtement lorsque le personnage qui le porte est déformé par interaction tactile

Virtual prototyping with surface reconstruction and freeform geometric modeling using level-set method

More and more products with complex geometries are being designed and manufactured by computer aided design (CAD) and rapid prototyping (RP) technologies. Freeform surface is a geometrical feature widely used in modern products like car bodies, airfoils and turbine blades as well as in aesthetic artifacts. How to efficiently design and generate digital prototypes with freeform surfaces is an important issue in CAD. This paper presents the development of a Virtual Sculpting system and addresses the issues of surface reconstruction from dexel data structures and freeform geometric modeling using the level-set method from distance field structure. Our virtual sculpting method is based on the metaphor of carving a solid block into a 3D freeform object using a 3D haptic input device integrated with the computer visualization. This dissertation presents the result of the study and consists primarily of four papers --Abstract, page iv

Evaluation of landscape architecture 3D modeling tools and practices in Finland

In Finland clients are increasingly demanding 3D models that are to be used in “combination models” during construction of infrastructure projects, which has led to a need for landscape architecture offices to acquire 3D modeling software. Especially older landscape architects do not have personal experience in doing 3D modeling, which is why the 3D modeling process, including the benefits and limitations, is not fully understood. Some research has already been done of the benefits and limitations of landscape architecture, but more research is needed on how these benefits and deficits apply to different 3D modeling software. This information can be useful for landscape architects and offices, who are beginning to include 3D modeling in their work process, and are wondering which software would best suit their needs. The goal of this Master’s thesis is to investigate how 3D modeling is done in Finland and how 3D modeling could better be used to benefit the needs of landscape architecture. In order to develop 3D modeling, the deficits of 3D modeling must first be found. After this it can be considered how to improve these deficits. The benefits of 3D modeling are also studied, because another way to develop 3D modeling is to increase the benefits. In order to find the currently known benefits and deficits, a literature review is conducted. The literature review outlines the knowledge gap, which helps to narrow down the research questions. Answers to the research questions are researched with a survey conducted with Finnish landscape architects and comparing 5 commonly used 3D modeling software in Finland in practice. In the conclusions the differences between these 5 software are outlined, and it is determined that the differences between 3D modeling software are largely explained by the target audience for those software. The development of 3D modeling requires software developers to acknowledge landscape architecture as one of the target audiences, for which plug-ins are one possible short-term solution. In the long term productivity would be increased by a BIM software developed for landscape architecture that could produce construction drawings and make better use of inital data. As the only university to teach landscape architecture in Finland, Aalto University has a high influence on which software future landscape architects use, so the university should check periodically if 3D modeling software that are best suited for landscape architecture are installed and included in teaching. Newly graduated landscape architects take their skills with them to offices and can suggest new software and work methods.Suomessa asiakkaat haluavat enenevässä määrin maisema-arkkitehtitoimistoilta 3D-malleja infrastruktuurihankkeissa käytettäviä yhdistelmätietomalleja varten, mikä on johtanut siihen, että maisema-arkkitehtitoimistojen täytyy ottaa 3D-mallinnusohjelmistoja käyttöönsä. Varsinkaan vanhemmilla maisema-arkkitehdeillä ei ole henkilökohtaista kokemusta 3D-mallintamisesta, minkä vuoksi 3D-mallinnuksen prosessia, etuja ja haittoja ei täysin ymmärretä. Joitain tutkimuksia on jo tehty maisema-arkkitehtuurin 3D-mallinnuksen eduista ja haitoista, mutta lisätutkimusta tarvitaan siitä, miten nämä todetut edut ja haitat pätevät eri 3D-mallinnusohjelmistoihin. Tämä tieto voi olla hyödyksi maisema-arkkitehdeille ja toimistoille, jotka alkavat sisällyttää 3D-mallintamista työprosessiinsa ja miettivät, mikä ohjelmisto sopisi tällä hetkellä parhaiten heidän tarpeisiinsa. Tämän diplomityön tavoite on tutkia, miten 3D-mallinnusta käytetään Suomessa ja millä keinoilla 3D-mallien hyödyntämistä voisi parantaa maisema-arkkitehtuurin tarpeita ajatellen. 3D-mallinnuksen kehittämiseksi tulee ensin selvittää, mitä ongelmia 3D-mallinnuksessa on. Vasta sitten voi harkita, miten ongelmia voisi parantaa. Myös 3D-mallinnuksen tuomat edut ovat yhtenä tutkimuksen kohteena, sillä toinen tapa kehittää 3D-mallinnusta on lisätä sen tuomia etuja. Nykyisin tiedossa olevien etujen ja haittojen selvittämiseksi suoritetaan kirjallisuustutkimus. Kirjallisuustutkimus auttaa artikuloimaan tutkimustiedosta puuttuvan aukon, minkä perusteella tutkimuskysymyksiä tarkennetaan. Näihin tutkimuskysymyksiin etsitään vastauksia suorittamalla tutkimuskysely suomalaisille maisema-arkkitehdeille ja tekemällä käytännön vertailu viidestä Suomessa yleisesti käytössä olevasta 3D-mallinnusohjelmasta. Johtopäätöksissä esitellään vertailtujen mallinnusohjelmien erot, ja todetaan ohjelmien välisten erojen selittyvän pitkälti 3D-mallinnusohjelmien kohderyhmien perusteella. 3D-mallinnuksen kehitys vaatii, että ohjelmistokehittäjät huomioivat maisema-arkkitehtuurin kohderyhmänä, mihin nykyisiin ohjelmistoihin kehitetyt lisäosat ovat yksi lyhyen tähtäimen ratkaisu. Pitkällä tähtäimellä työskentelyä tehostaisi maisema-arkkitehtuurin tarpeisiin kehitetty BIM-ohjelmisto, joka pystyy tuottamaan rakennepiirroksia ja hyödyntämään lähtötietoja nykyistä tehokkaammin. Ainoana maisema-arkkitehtuuria opettavana yliopistona Suomessa Aalto-yliopiston opetuksella on suuri vaikutus käytettäviin ohjelmiin, joten yliopiston tulisi tarkistaa säännöllisesti, onko maisema-arkkitehtuurin 3D-mallinnukseen parhaiten soveltuvia ohjelmistoja asennettuna ja sisällytettynä opetukseen. Vastavalmistuneet maisema-arkkitehdit vievät osaamisensa mukanaan toimistoihin ja voivat ehdottaa uusia ohjelmistoja ja työskentelymenetelmiä