Virtual Reality Interfaces for Product Design: Finding User Interface solutions for design creation within Virtual Reality

The focus of Virtual Reality has gone from research to widespread adoption in entertainment and practical directions, like automotive design and architectural visualization. With that, we have to take into consideration the best way to give in-experience control to the user and the interaction within the interface. Recent studies explore the ergonomic considerations and zones of content for VR interfaces. But Virtual Reality interaction design has a long way to go and nowadays is done mainly like a projection of 2D screens, with planar interfaces in the 3D space, almost ignoring the immersive potential of the Virtual Reality medium (Alger 2015; Google Developers 2017). Designers that work with 3D objects might find it difficult to make design decisions and validate their concepts based on context and empathy. To help with this, they often prototype, which can take a great deal of time and effort. Virtual reality can be a tool that improves the process and gives the designer an unconstrained and flexible canvas. By reimagining interactions for Virtual Reality, this thesis aims to create interface tools that help designers explore shape and manipulate their designs

Molecular manipulation in augmented reality: A user experience design applied research on new paradigms of interaction.

This thesis presents the further developments of VR CHEM (2017), a molecular manipulation software in Virtual Reality. After evaluating a shift to Augmented Reality, changing consequently the name to AR CHEM, the project focused on this technology as it offers a competitive advantage for future development, especially considering the less invasive user experience and the incoming investments in the market. The project is developed using HoloLens and Unity 3D as a platform, as these tools are considered the state of the art for AR technologies (2018). The thesis presents a first iteration of new paradigms of interactions in AR: the 3D user interfaces and the Platform System. While the first explore the possibility of giving information to the user through 3D models used as UI/UX indications, the second strengthen the user’s visual perception of depth in space, tying all the content to more tangible elements

Direct modeling techniques in the conceptual design stage in immersive environments for DfA&D

Due to the fast – growing competition of the mass – products markets, companies are looking for new technologies to maximize productivity and minimize time and costs. In the perspective of Computer Aided Process Planning (CAPP), companies want to optimize fixture design and assembly planning for different goals. To meet these demands, the designers' interest in Design for Assembly and Disassembly is growing considerably and is increasingly being integrated into the CAPP. The work described in this thesis aims to exploit immersive technologies to support the design of mating elements and assembly / disassembly, by developing a data exchange flow between the immersive environment and the modeling environment that provides the high – level modeling rules, both for modeling features and for assembly relationships. The main objective of the research is to develop the capability to model and execute simple coupling commands in a virtual environment by using fast direct modeling commands. With this tool the designer can model the coupling elements, position them and modify their layout. Thanks to the physical engine embedded in the scene editor software, it is possible to take into consideration physical laws such as gravity and collision between elements. A library of predefined assembly features has been developed through the use of an external modeling engine and put into communication with the immersive interaction environment. Subsequently, the research involved the study of immersive technologies for workforce development and training of workers. The research on immersive training involved industrial case studies, such as the projection of the disassembly sequence of an industrial product on a head mounted display, and less industrial case studies, such as the manual skills development of carpenters for AEC sectors and the surgeon training in the pre – operative planning in medical field

Hand interaction designs in mixed and augmented reality head mounted display: a scoping review and classification

Mixed reality has made its first step towards democratization in 2017 with the launch of a first generation of commercial devices. As a new medium, one of the challenges is to develop interactions using its endowed spatial awareness and body tracking. More specifically, at the crossroad between artificial intelligence and human-computer interaction, the goal is to go beyond the Window, Icon, Menu, Pointer (WIMP) paradigm humans are mainly using on desktop computer. Hand interactions either as a standalone modality or as a component of a multimodal modality are one of the most popular and supported techniques across mixed reality prototypes and commercial devices. In this context, this paper presents scoping literature review of hand interactions in mixed reality. The goal of this review is to identify the recent findings on hand interactions about their design and the place of artificial intelligence in their development and behavior. This review resulted in the highlight of the main interaction techniques and their technical requirements between 2017 and 2022 as well as the design of the Metaphor-behavior taxonomy to classify those interactions

Augmented reality and its aspects: a case study for heating systems

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáThanks to the advances of technology in various domains, and the mixing between real and virtual worlds. Allowed this master’s thesis to explore concepts related to virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR). The development and comparison of Android applications and Microsoft HoloLens aimed to solve a deadlock in the recognition of instructions by the users. We used an interactive manual of assembly and disassembly for taps of residential heaters. Therefore, this work deals with three main parts. Firstly, the exploration of the concepts of VR, AR, MR, and XR. Secondly, 3D modeling and animations techniques. Finally, the development of applications using Vuforia, Wikitude, and MRTK. The users tried our application “HeaterGuideAR” to verify the effectiveness of the instruction passed by the interactive manual. Only a few users had some difficulties at the beginning of the trials. Thus, it was necessary to provide aid tools. However, other users were able to disassemble the faucet without any external help. We suggest continuing this work with more explorations, models, and situations.Graças aos últimos avanços tecnológicos em diversas áreas deram a possibilidade de fazer a mistura do mundo real com o virtual. É com este intuito que esta tese de mestrado veio expor os conceitos relacionados à realidade virtual (RV), realidade aumentada (RA), realidade mista (RM) e realidade estendida (RE). O desenvolvimento e comparação de aplicativos Android e Microsoft HoloLens teve como objetivo resolver um impasse no entendimento de instruções por parte dos usuários. Utilizamos um manual interativo para montagem e desmontagem de torneiras de aquecedores residenciais. Este trabalho, portanto, lida com três partes principais. Na primeira, a exploração dos conceitos de RV, RA, RM e RE. Na segunda, modelagem 3D e técnicas de animações. E por fim, o desenvolvimento de aplicações usando Vuforia, Wikitude e MRTK. A aplicação “HeaterGuideAR” foi testada pelos usuários afim de verificar a eficácia da instrução passada pelo manual interativo. Apenas alguns usuários tiveram algumas dificuldades no início dos testes. Sendo que, foi necessário fornecer algumas ferramentas de auxílio. Mesmo assim, outros usuários conseguiram desmontar a torneira sem ajuda externa. Sugerimos continuar este trabalho com mais explorações, modelos e situações.Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do Paran

Heads Up! Supporting Maritime Navigation using Augmented Reality

Augmented Reality (AR) is a technology that shows potential for the improvement of maritime safety. Today, the ship bridge suffers from a lack of standardization and integration. Head-Mounted Displays (HMDs) may alleviate these challenges by showing information when relevant and enhancing operator mobility. Microsoft HoloLens 2 (HL2) is such a HMD. Prior research shows the potential of HMDs in the Maritime AR domain (Rowen et al., 2019). Limited research has been conducted however on the design of AR User Interfaces (UIs) for maritime applications leveraging HMDs. As a result, no framework exists to test new UI designs in the real world, which is necessary due to many variables that cannot be accurately modelled in a lab setting. This led to the research questions (RQs) 1. What makes an effective head-mounted AR UI for maritime navigation? (RQ1); and 2. How can HL2 be used as a ship bridge system? (RQ2) A Research through Design (RtD) process is detailed where a UI design and functional prototype was developed in collaboration with end-users. The prototype, named Sjør, implements the aforementioned interface, provides a framework for in-context UI testing and can be viewed as the next step towards standardizing AR UIs for the maritime industry. The design and development process led to three contributions to the Maritime AR domain. Firstly, a framework for the visualization of location-based data about points of interest on predefined canvases co-located in the real world was developed (Technology Readiness Level (TRL) 6), which runs on the HL2. This first contribution is defined in Section 4 and provides an answer to RQ2. Secondly, using this framework, an interface design (including interactions) is developed in collaboration with end-users and proposed as an answer to RQ1. This process is described in Section 5. The third contribution is a research agenda which provides insights into how contemporary and future research can leverage the developed framework. Section 7 discloses this research agenda.Master's Thesis in Interaction and Media DesignMIX350MASV-MI

DXR: A Toolkit for Building Immersive Data Visualizations

This paper presents DXR, a toolkit for building immersive data visualizations based on the Unity development platform. Over the past years, immersive data visualizations in augmented and virtual reality (AR, VR) have been emerging as a promising medium for data sense-making beyond the desktop. However, creating immersive visualizations remains challenging, and often require complex low-level programming and tedious manual encoding of data attributes to geometric and visual properties. These can hinder the iterative idea-to-prototype process, especially for developers without experience in 3D graphics, AR, and VR programming. With DXR, developers can efficiently specify visualization designs using a concise declarative visualization grammar inspired by Vega-Lite. DXR further provides a GUI for easy and quick edits and previews of visualization designs in-situ, i.e., while immersed in the virtual world. DXR also provides reusable templates and customizable graphical marks, enabling unique and engaging visualizations. We demonstrate the flexibility of DXR through several examples spanning a wide range of applications