Leveraging eXtented Reality & Human-Computer Interaction for User Experi- ence in 360◦ Video

EXtended Reality systems have resurged as a medium for work and entertainment. While 360o video has been characterized as less immersive than computer-generated VR, its realism, ease of use and affordability mean it is in widespread commercial use. Based on the prevalence and potential of the 360o video format, this research is focused on improving and augmenting the user experience of watching 360o video. By leveraging knowledge from Extented Reality (XR) systems and Human-Computer Interaction (HCI), this research addresses two issues affecting user experience in 360o video: Attention Guidance and Visually Induced Motion Sickness (VIMS). This research work relies on the construction of multiple artifacts to answer the de- fined research questions: (1) IVRUX, a tool for analysis of immersive VR narrative expe- riences; (2) Cue Control, a tool for creation of spatial audio soundtracks for 360o video, as well as enabling the collection and analysis of captured metrics emerging from the user experience; and (3) VIMS mitigation pipeline, a linear sequence of modules (including optical flow and visual SLAM among others) that control parameters for visual modi- fications such as a restricted Field of View (FoV). These artifacts are accompanied by evaluation studies targeting the defined research questions. Through Cue Control, this research shows that non-diegetic music can be spatialized to act as orientation for users. A partial spatialization of music was deemed ineffective when used for orientation. Addi- tionally, our results also demonstrate that diegetic sounds are used for notification rather than orientation. Through VIMS mitigation pipeline, this research shows that dynamic restricted FoV is statistically significant in mitigating VIMS, while mantaining desired levels of Presence. Both Cue Control and the VIMS mitigation pipeline emerged from a Research through Design (RtD) approach, where the IVRUX artifact is the product of de- sign knowledge and gave direction to research. The research presented in this thesis is of interest to practitioners and researchers working on 360o video and helps delineate future directions in making 360o video a rich design space for interaction and narrative.Sistemas de Realidade EXtendida ressurgiram como um meio de comunicação para o tra- balho e entretenimento. Enquanto que o vídeo 360o tem sido caracterizado como sendo menos imersivo que a Realidade Virtual gerada por computador, o seu realismo, facili- dade de uso e acessibilidade significa que tem uso comercial generalizado. Baseado na prevalência e potencial do formato de vídeo 360o, esta pesquisa está focada em melhorar e aumentar a experiência de utilizador ao ver vídeos 360o. Impulsionado por conhecimento de sistemas de Realidade eXtendida (XR) e Interacção Humano-Computador (HCI), esta pesquisa aborda dois problemas que afetam a experiência de utilizador em vídeo 360o: Orientação de Atenção e Enjoo de Movimento Induzido Visualmente (VIMS). Este trabalho de pesquisa é apoiado na construção de múltiplos artefactos para res- ponder as perguntas de pesquisa definidas: (1) IVRUX, uma ferramenta para análise de experiências narrativas imersivas em VR; (2) Cue Control, uma ferramenta para a criação de bandas sonoras de áudio espacial, enquanto permite a recolha e análise de métricas capturadas emergentes da experiencia de utilizador; e (3) canal para a mitigação de VIMS, uma sequência linear de módulos (incluindo fluxo ótico e SLAM visual entre outros) que controla parâmetros para modificações visuais como o campo de visão restringido. Estes artefactos estão acompanhados por estudos de avaliação direcionados para às perguntas de pesquisa definidas. Através do Cue Control, esta pesquisa mostra que música não- diegética pode ser espacializada para servir como orientação para os utilizadores. Uma espacialização parcial da música foi considerada ineficaz quando usada para a orientação. Adicionalmente, os nossos resultados demonstram que sons diegéticos são usados para notificação em vez de orientação. Através do canal para a mitigação de VIMS, esta pesquisa mostra que o campo de visão restrito e dinâmico é estatisticamente significante ao mitigar VIMS, enquanto mantem níveis desejados de Presença. Ambos Cue Control e o canal para a mitigação de VIMS emergiram de uma abordagem de Pesquisa através do Design (RtD), onde o artefacto IVRUX é o produto de conhecimento de design e deu direcção à pesquisa. A pesquisa apresentada nesta tese é de interesse para profissionais e investigadores tra- balhando em vídeo 360o e ajuda a delinear futuras direções em tornar o vídeo 360o um espaço de design rico para a interação e narrativa

DESIGNING AN EXTENDED REALITY MOBILE GUIDE APPLICATION TO COMMUNICATE AND INTERPRET SERIAL WORLD HERITAGE SITES: A CASE STUDY OF KOREA’S SEOWON UNESCO WORLD HERITAGE SITE

Extended reality (XR) mobile guide applications offer unprecedented potential for immersive visitor experiences and in-depth knowledge retention to promote cultural learning at large-scale heritage sites, but, despite their significant development, the literature underexplores these applications for World Heritage sites, especially the serial properties that are spatially dispersed in various locations but configured as a single property. This paper describes a framework (blueprint) for the development of an XR mobile guide application focused on serial properties. By incorporating interactive XR and edutainment features, it explores a way to comprehensively reveal the interconnections between the heritage attributes of the subcomponent of the serial properties and their larger, cross-cultural context vis-à-vis the Outstanding Universal Values. To this end, at the Seowon, the UNESCO World Heritage site in Korea, we analyzed user interactions with a prototype of the XR application to identify user preferences and areas for improving the framework

Lecture Notes on Interactive Storytelling

These lecture notes collect the material used in the advanced course 'Interactive Storytelling' organized biannually at the Department of Future Technologies, University of Turku, Finland. Its aim is to present the key concepts behind interactive digital storytelling (IDS) as well as to review proposed and existing IDS systems. The course focuses on the four partakers of IDS: the platform, the designer, the interactor, and the storyworld. When constructing a platform, the problem is to select an appropriate approach from tightly controlled to emergent storytelling. On this platform, the designer is then responsible for creating the content (e.g., characters, props, scenes and events) for the storyworld, which is then experienced and influenced by the interactor. The structure and relationships between these partakers is explained from a theoretical perspective as well as using existing IDS systems as examples.</p

Lights, camera, interaction! Interactive Film and its Transformative Potential.

This thesis concentrates on the emerging field of interactive film. Digital interactive and networked media offer so many possibilities to create stories that it is necessary to define what an interactive film is and whether there is any continuity with the linear film form. This thesis explores whether interactive narratives in the form of interactive film have the potential to offer a transformative learning experience regarding societal and political topics. Butterfly is an interactive short film that uses a second-screen technique to raise awareness of the dangers of cyberbullying. As a case study for a potential transformative experience, the film is described and evaluated by means of interactive screenings and a user experience study. Findings show there is definite potential for interactive films to create strong emotions in users and to possibly produce a transformative experience with educational implications. Keywords: Transformative design, Interactive Narrative Design, Interactive film, User Experience Evaluation

Narrativas Interativas Como Linguagem do Jornalismo

O ciberjornalismo mundial já está consolidado como principal fonte de informação no mundo, porém, as redações destes veículos, ainda buscam formatos e linguagens com as quais possam informar e fidelizar sua audiência, alcançando, por consequência, maior sustentabilidade econômica. É neste aspecto que esta investigação realizou a publicação de uma reportagem em formato de Narrativas Interativas em parceria com um veículo nativo digital de nicho, e assim compreendeu, através da análise de navegação e inquérito com leitor/espectador/ouvintes, que a alteração da linguagem e formato da informação influem na decisão do usuário em permanecer ou não fiel a um meio de comunicação. Estes dados, bem como o aprodunfamento bibliográfico da adaptação do jornalismo a novas plataformas tecnológicas por esta investigação apresentados, dão mecanismos para os meios de comunicação poossam projetar seus produtos, bem como para a academia antecipar tendências de comunicação para o ciberespaço.The cyberjournalism is already consolidated as the main source of information in the worldwide, however, the newsrooms of these medias still seek formats and languages with which they can inform and retain their audience, thus achieving greater economic sustainability. It is in this aspect, this investigation carried out the publication of a report in Narratives Interactives' format in partnership with a niche digital native media, and thus understood, through navigation analysis and survey with users, that the change of the format and language of a article influences the user's decision to remain faithful or not to a means of communication. These data, as well as the bibliographic production of journalism's adaptation to new technological platforms presented by this investigation, provide mechanisms for the media to design their products, as well as for the academy to anticipate communication trends for cyberspace

Políticas de Copyright de Publicações Científicas em Repositórios Institucionais: O Caso do INESC TEC

A progressiva transformação das práticas científicas, impulsionada pelo desenvolvimento das novas Tecnologias de Informação e Comunicação (TIC), têm possibilitado aumentar o acesso à informação, caminhando gradualmente para uma abertura do ciclo de pesquisa. Isto permitirá resolver a longo prazo uma adversidade que se tem colocado aos investigadores, que passa pela existência de barreiras que limitam as condições de acesso, sejam estas geográficas ou financeiras. Apesar da produção científica ser dominada, maioritariamente, por grandes editoras comerciais, estando sujeita às regras por estas impostas, o Movimento do Acesso Aberto cuja primeira declaração pública, a Declaração de Budapeste (BOAI), é de 2002, vem propor alterações significativas que beneficiam os autores e os leitores. Este Movimento vem a ganhar importância em Portugal desde 2003, com a constituição do primeiro repositório institucional a nível nacional. Os repositórios institucionais surgiram como uma ferramenta de divulgação da produção científica de uma instituição, com o intuito de permitir abrir aos resultados da investigação, quer antes da publicação e do próprio processo de arbitragem (preprint), quer depois (postprint), e, consequentemente, aumentar a visibilidade do trabalho desenvolvido por um investigador e a respetiva instituição. O estudo apresentado, que passou por uma análise das políticas de copyright das publicações científicas mais relevantes do INESC TEC, permitiu não só perceber que as editoras adotam cada vez mais políticas que possibilitam o auto-arquivo das publicações em repositórios institucionais, como também que existe todo um trabalho de sensibilização a percorrer, não só para os investigadores, como para a instituição e toda a sociedade. A produção de um conjunto de recomendações, que passam pela implementação de uma política institucional que incentive o auto-arquivo das publicações desenvolvidas no âmbito institucional no repositório, serve como mote para uma maior valorização da produção científica do INESC TEC.The progressive transformation of scientific practices, driven by the development of new Information and Communication Technologies (ICT), which made it possible to increase access to information, gradually moving towards an opening of the research cycle. This opening makes it possible to resolve, in the long term, the adversity that has been placed on researchers, which involves the existence of barriers that limit access conditions, whether geographical or financial. Although large commercial publishers predominantly dominate scientific production and subject it to the rules imposed by them, the Open Access movement whose first public declaration, the Budapest Declaration (BOAI), was in 2002, proposes significant changes that benefit the authors and the readers. This Movement has gained importance in Portugal since 2003, with the constitution of the first institutional repository at the national level. Institutional repositories have emerged as a tool for disseminating the scientific production of an institution to open the results of the research, both before publication and the preprint process and postprint, increase the visibility of work done by an investigator and his or her institution. The present study, which underwent an analysis of the copyright policies of INESC TEC most relevant scientific publications, allowed not only to realize that publishers are increasingly adopting policies that make it possible to self-archive publications in institutional repositories, all the work of raising awareness, not only for researchers but also for the institution and the whole society. The production of a set of recommendations, which go through the implementation of an institutional policy that encourages the self-archiving of the publications developed in the institutional scope in the repository, serves as a motto for a greater appreciation of the scientific production of INESC TEC

ICS Materials. Towards a re-Interpretation of material qualities through interactive, connected, and smart materials.

The domain of materials for design is changing under the influence of an increased technological advancement, miniaturization and democratization. Materials are becoming connected, augmented, computational, interactive, active, responsive, and dynamic. These are ICS Materials, an acronym that stands for Interactive, Connected and Smart. While labs around the world are experimenting with these new materials, there is the need to reflect on their potentials and impact on design. This paper is a first step in this direction: to interpret and describe the qualities of ICS materials, considering their experiential pattern, their expressive sensorial dimension, and their aesthetic of interaction. Through case studies, we analyse and classify these emerging ICS Materials and identified common characteristics, and challenges, e.g. the ability to change over time or their programmability by the designers and users. On that basis, we argue there is the need to reframe and redesign existing models to describe ICS materials, making their qualities emerge

Alfabetización digital en la educación: posicionamientos ausentes de una definición consensuada

In a world where the introduction of technologies in social interaction environments has generated important transformations in the way people communicate, acquire knowledge, and share information. Digital literacy (DA) establishes a fundamental role in universal education, so higher education is no exception. Because it involves not only initial knowledge of how to use technological devices and tools, but also the ability to use, understand and critically evaluate digital technology effectively and safely. The objective of this research is to analyze the theoretical contributions about AD in higher education students in the period from 2018 to 2023 to generate an approach to its meaning. The methodology employs a systematic literature review approach to identify existing knowledge on the topic, as well as areas not yet explored. Specific inclusion and exclusion criteria are used in the Google Scholar database to carry out the search for articles. As a third result, eight open access articles obtained from two previous reviews were examined, the first with 108 articles and the second with 68. Inaccuracies and lack of knowledge regarding the concept were recorded. They generally refer to the AD construct as digital competencies or skills. Regarding the findings, the absence of scientific production related to the topic is identified, with the majority being international contributions. The above makes its adoption, development, and promotion within the university community difficult