Nabia: virtual reality movement exploration and choreographic tool

Dissertação de mestrado apresentada à Escola Superior de Comunicação Social como parte dos requisitos para obtenção de grau de mestre em Audiovisual e Multimédia.As tecnologias de Realidade Virtual e Aumentada têm ganho uma maior relevância no mercado com o aparecimento de soluções acessiveis ao consumidor geral, como o Meta Quest 2. Os videojogos têm dominado as soluções XR, mas, com a democratização do acesso a ferramentas de Realidade Virtual e Aumentada, existe uma crescente procura pelo uso de XR para soluções fora do entretenimento. A área de Design de Interação é assim, como no desenvolvimento de soluções web e soluções para mobile, uma componente fulcral para o desenvolvimento da Realidade Virtual e Aumentada e na definição das normas e heurísticas de XR Design. A inovação técnológica e o desenvolvimento da arte sempre se relacionam com proximidade. O aparecimento do cinema através do desenvolvimento técnologico, o desenvolvimento de musica eletrónica pela invenção dos sintetizadores são exemplos dessa mesma proximidade. O cruzamento entre tecnologias de Realidade Virtual e Aumentada com a arte aparenta ser um passo natural do desenvolvimento de ambas as vertentes. Assim, propõe-se o desenvolvimento de uma solução VR, que procure explorar e definir formas de interação entre o ambiente virtual e a linguagem de movimento da área artistica da Dança. Desenvolveu-se então, como objeto de projeto final, uma ferramenta de Realidade Virtual, designada de Nabia, que permite aos utilizadores explorarem e comporem o seu movimento, através da interação e inspiração com o meio virtual.ABSTRACT: Virtual Reality and Augmented Reality technologies have gained greater prominence in the market with the emergence of accessible solutions for the general consumer, such as the Meta Quest 2. Video games have dominated XR solutions, but with the democratization of access to Virtual and Augmented Reality tools, there is a growing demand for the use of XR in non-entertainment solutions. The field of Interaction Design, as in the development of web and mobile solutions, is a crucial component in the development of Virtual and Augmented Reality and in defining XR Design standards and heuristics. Technological innovation and the development of art have always been closely related. The emergence of cinema through technological development, the development of electronic music through the invention of synthesizers are examples of this proximity. The intersection of Virtual and Augmented Reality technologies with art appears to be a natural step in the development of both aspects. Therefore, it is proposed the development of a VR solution that seeks to explore and define forms of interaction between the virtual environment and the movement language of the artistic field of Dance. It was developed a Virtual Reality tool, named Nabia, as the final project, which allows users to explore and compose their movement through interaction and inspiration from the virtual environment.N/

Game vaporware as design fictions

In this research we examine games, and games hardware, that can be classed as ‘Vaporware’. More specifically software that was never written, or hardware that was never built, and consequently no one ever played. In particular we are considering such vaporware as examples of ‘Design Fiction’ as they once represented speculative visions of the future based on emerging technology. Vaporware is a term generally used to describe products that are announced to the general public but are never actually manufactured. Whereas design fiction is a term used to describe plausible ‘diegetic prototypes’ that are built, or suggested, to create an opportunity for discourse about possible technological futures. Whilst it could be argued vaporware games are simply failed products that were justifiably scrapped before joining the long lists of come-to-nothing games and consoles, by reviewing examples we offer an alternative view that they can serve as objects of discourse for exposing the potential futures of video games and thus could be considered in terms of design fiction. To add further weight to the argument that games can be useful as design fictions we then consider “Game of Drones”, an example of a design fiction that pivots around a game element, to illustrate how the deliberate use of design fiction can stimulate discourse around game futures (in this case the growing promotion of ‘gamified’ services as means of engaging users). Whilst the notion of designing games that will never be built may seem paradoxical in relation to the Games industry’s predominantly commercial aims, we believe that the deliberate adoption of design fiction as a practice within game design would facilitate the emergence of meaningful discussions around future gaming without the frustrations induced by vaporware

A sound idea: An investigation into accessible video game design for the deaf and hard of hearing

A widely accepted, and incorrect, assumption towards hearing accessibility in video games is that deaf and hard of hearing (DHH) users are those who encounter the least barriers and are generally well catered for. Rapid advancement in video game technology has seen video game sound evolve from simple blips generated by internal circuitry to fully realised digital audio used to convey critical information. To accommodate the DHH, this information needs to be conveyed in an alternative manner. However, evidence suggests existing accessible design solutions for the DHH lack specificity and are insufficient. Thus, the inability to hear, or hear well, has historically resulted in DHH users left with impeded experience and gameplay. This thesis describes an investigation to address the primary research question: How might accessible video game design practices be facilitated to better accommodate the deaf and hard of hearing? To examine this question, an action research method as part of a transformative mixed methods methodology was used. Data collection procedures included critical analysis of literature, observations, and a cross-sectional self-administered survey for triangulation. The critical analysis of literature exposed issues relating to accessible video game design, particularly in relation to the identification of solutions and technical implementation. Further, issues related to the classification of video game software were identified. This posed potential problems with identification of game design methods and led to the development of a new video game classification model. The new model informed an analysis on the methods used for the design of video games, and results were visually represented and mapped to the different approaches to accessible design. Subsequent analysis determined that a game assessment framework is a suitable approach to facilitating accessible design. Further investigation identified visual feedback as the most suitable form of complementary feedback to game audio. This led to the development of a new model to classify visual feedback elements used in video games, and identification of audio feedback categories based on diegetic film theory. Through triangulation of results, a new game feedback model (GFM) was developed. The GFM was used for observational experimentation to identify and classify individual visual feedback elements used in video games. Each element was analysed and mapped to categories of game sound. The resulting model, with populated data, was used to determine what visual feedback elements may be used to complement specific categories of critical game audio. A survey was subsequently used for triangulation, and resulted in amendments to the final model. Through iterative development, and interpretation of findings, the research culminated in the development of a game assessment framework. The three-step framework aids in the classification of game sounds; assesses the impact of those game sounds; and provides recommendations for complementary visual feedback elements for sounds identified as having an adverse impact on user experience and gameplay if they were to be removed. The framework is innovative and has the potential to provide practical guidance for developers of video games. In addition, this research provides the foundation for future research, with the potential to influence accessible game design for the DHH

Exploiting fashion x-commerce through the empowerment of voice in the fashion virtual reality arena. Integrating voice assistant and virtual reality technologies for fashion communication

The ongoing development of eXtended Reality (XR) technologies is supporting a rapid increase of their performances along with a progressive decrease of their costs, making them more and more attractive for a large class of consumers. As a result, their widespread use is expected within the next few years. This may foster new opportunities for e-commerce strategies, giving birth to an XR-based commerce (x-commerce) ecosystem. With respect to web and mobile-based shopping experiences, x-commerce could more easily support brick-and-mortar store-like experiences. One interesting and consolidated one amounts to the interactions among customers and shop assistants inside fashion stores. In this work, we concentrate on such aspects with the design and implementation of an XR-based shopping experience, where vocal dialogues with an Amazon Alexa virtual assistant are supported, to experiment with a more natural and familiar contact with the store environment. To verify the validity of such an approach, we asked a group of fashion experts to try two different XR store experiences: with and without the voice assistant integration. The users are then asked to answer a questionnaire to rate their experiences. The results support the hypothesis that vocal interactions may contribute to increasing the acceptance and comfortable perception of XR-based fashion shopping

Interactive Experience Design: Integrated and Tangible Storytelling with Maritime Museum Artefacts

Museums play the role of intermediary between cultural heritage and visitors, and are often described as places and environments for education and enjoyment. The European Union also encourages innovative uses of museums to support education through the cultural heritage resources. However, the importance of visitors’ active role in museums as places for education and entertainment, on the one hand, and the growing and indispensable presence of technology in the cultural heritage domain, on the other hand, provided the initial ideas to develop the research. This thesis, presents the study and design for an interactive storytelling installation for a maritime museum. The installation is designed to integrate different museum artefacts into the storytelling system to enrich the visitors experience through tangible storytelling. The project was conducted in collaboration with another PhD student, Luca Ciotoli. His contribution was mainly focused on the narrative and storytelling features of the research, while my contribution was focused on the interaction- and technology-related features, including the design and implementation of the prototype. The research is deployed using a four-phase iterative approach. The first phase of the research, Study, deals with literature review and different studies to identify the requirements. The second phase, Design, determines the broad outlines of the project i.e., an interactive storytelling installation. The design phase includes interaction and museum experience design. We investigated different design approaches, e.g., interaction and museum experience design, to develop a conceptual design. The third phase, prototype, allows us to determine how to fulfill the tasks and meet the requirements that are established for the research. Prototyping involves content creation, storyboarding, integrating augmented artefacts into the storytelling system. Th final phase, test, refers to the evaluations that are conducted during the aforementioned phases e.g., formative and the final usability testing with users. The outcome of the research confirms previous results in the literature about how digital narratives can be enriched with the tangible dimension, moreover it shows how this dimension can enable to communicate stories and knowledge of the past that are complex, such as the art of navigating in the past, by integrating tangible objects that play different roles in the storytelling process

Intentional Friction in the User Interface of Digital Games

Embora projetar fricção intencionalmente em interfaces do utilizador de jogos possa ser uma estratégia adequada para desafiar as crenças dos jogadores e estimular a reflexão, as práticas convencionais de interface do utilizador são frequentemente influenciadas por um paradigma abrangente de facilidade de uso e prazer. Esta dissertação investiga como designers podem empregar fricção intencional em interfaces do utilizador de jogos digitais para criar experiências significativas e inspirar reflexão nos seus jogadores. Primeiro, revimos a literatura para enquadrar o que constitui elementos de interface no game design, o paradigma de usabilidade e diversão e outras perspetivas que oferecem contexto para o uso da fricção como estratégia. Depois, exploramos instâncias de jogos que usam fricção na interface do utilizador quando apropriado como estratégia para expressar um ponto de vista, desafiar sistemas atuais ou fomentar a reflexão crítica. O ponto de partida para nossas observações são os sete princípios de design de Donald Norman e as heurísticas de usabilidade de Jakob Nielsen. Como resultado, identificamos seis estratégias de fricção intencional distintas. Em seguida, realizamos duas sessões de workshop de co-criação com um total de sete participantes com experiência em interface de utilizador ou design de jogos para identificar estratégias e perspetivas adicionais. As estratégias coletadas foram reunidas numa ferramenta de cartas. Por fim, realizamos uma sessão inicial de validação da ferramenta com quatro participantes com resultados promissores, sugerindo que as estratégias de fricção da ferramenta conseguiram impulsionar a expressividade como um componente importante do processo de discussão e ideação dos participantes. Embora este trabalho não esteja focado em coletar todas as abordagens de design de fricção indiscriminadamente, as estratégias identificadas sugerem técnicas mais subtis do que apenas enquadrar em reverso os princípios para criar um design amigável.  While intentionally designing friction in gaming user interfaces may be a suitable strategy for challenging players' beliefs and prompting reflection, conventional user interface practices are frequently influenced by an overarching paradigm of user-friendliness and enjoyment. This dissertation investigates how designers might employ intentional friction in digital game user interfaces to create meaningful experiences and inspire reflection in its players. First, we review the literature to frame what constitutes interface elements in game design, the user-friendly and enjoyment paradigm, and other perspectives that offer context to using friction as a strategy. Afterward, we explore game instances that use user interface friction when appropriate as a strategy to express a point of view, to challenge current systems, or to foment critical reflection. The starting point for our observations is Donald Norman's seven design principles and Jakob Nielsen's usability heuristics. As a result, we identify six distinct intentional friction strategies. Next, we ran two co-creation workshop sessions with a total of seven participants with user interface or game design backgrounds to identify additional strategies and perspectives. The strategies gathered were collected in a deck-based tool. Finally, we ran an initial tool validation session with four participants with promising results, suggesting that the tool's friction strategies were able to drive expressiveness as an important component of the participant's discussion and ideation process. Although this work is not focused on collecting all friction design approaches indiscriminately, the identified strategies suggest more nuanced techniques than just framing the principles to create a friendly design in reverse

VR Lab: User Interaction in Virtual Environments using Space and Time Morphing

Virtual Reality (VR) allows exploring changes in space and time that would otherwise be difficult to simulate in the real world. It becomes possible to transform the virtual world by increasing or diminishing distances or playing with time delays. Analysing the adaptability of users to different space-time conditions allows studying human perception and finding the right combination of interaction paradigms. Different methods have been proposed in the literature to offer users intuitive techniques for navigating wide virtual spaces, even if restricted to small physical play areas. Other studies investigate latency tolerance, suggesting humans’ inability to detect slight discrepancies between visual and proprioceptive sensory information. These studies contribute valuable insights for designing immersive virtual experiences and interaction techniques suitable for each task. This dissertation presents the design, implementation, and evaluation of a tangible VR Lab where spatiotemporal morphing scenarios can be studied. As a case study, we restricted the scope of the research to three spatial morphing scenarios and one temporal morphing scenario. The spatial morphing scenarios compared Euclidean and hyperbolic geometries, studied size discordance between physical and virtual objects, and the representation of hands in VR. The temporal morphing scenario investigated from what visual delay the task performance is affected. The users’ adaptability to the different spatiotemporal conditions was assessed based on task completion time, questionnaires, and observed behaviours. The results revealed significant differences between Euclidean and hyperbolic spaces. They also showed a preference for handling virtual and physical objects with concordant sizes, without any virtual representation of the hands. Although task performance was affected from 200 ms onwards, participants considered the ease of the task to be affected only from 500 ms visual delay onwards.A Realidade Virtual (RV) permite explorar mudanças no espaço e no tempo que de outra forma seriam difíceis de simular no mundo real. Torna-se possível transformar o mundo virtual aumentando ou diminuindo as distâncias ou manipulando os atrasos no tempo. A análise da adaptabilidade dos utilizadores a diferentes condições espaço-temporais permite estudar a perceção humana e encontrar a combinação certa de paradigmas de interação. Diferentes métodos têm sido propostos na literatura para oferecer aos utilizadores técnicas intuitivas de navegação em espaços virtuais amplos, mesmo que restritos a pequenas áreas físicas de jogo. Outros estudos investigam a tolerância à latência, sugerindo a incapacidade do ser humano de detetar ligeiras discrepâncias entre a informação sensorial visual e propriocetiva. Estes estudos contribuem com valiosas informações para conceber experiências virtuais imersivas e técnicas de interação adequadas a cada tarefa. Esta dissertação apresenta o desenho, implementação e avaliação de um Laboratório de RV tangível onde podem ser estudados cenários de distorção espaço-temporal. Como estudo de caso, restringimos o âmbito da investigação a três cenários de distorção espacial e um cenário de distorção temporal. Os cenários de distorção espacial compararam geometrias Euclidianas e hiperbólicas, estudaram a discordância de tamanho entre objetos físicos e virtuais, e a representação das mãos em RV. O cenário de distorção temporal investigou a partir de que atraso visual o desempenho da tarefa é afetado. A adaptabilidade dos utilizadores às diferentes condições espaço-temporais foi avaliada com base no tempo de conclusão da tarefa, questionários, e comportamentos observados. Os resultados revelaram diferenças significativas entre os espaços Euclidiano e hiperbólico. Também mostraram a preferência pelo manuseamento de objetos virtuais e físicos com tamanhos concordantes, sem qualquer representação virtual das mãos. Embora o desempenho da tarefa tenha sido afetado a partir dos 200 ms, os participantes consideraram que a facilidade da tarefa só foi afetada a partir dos 500 ms de atraso visual