'Breaking the glass': preserving social history in virtual environments

New media technologies play an important role in the evolution of our society. Traditional museums and heritage sites have evolved from the ‘cabinets of curiosity’ that focused mainly on the authority of the voice organising content, to the places that offer interactivity as a means to experience historical and cultural events of the past. They attempt to break down the division between visitors and historical artefacts, employing modern technologies that allow the audience to perceive a range of perspectives of the historical event. In this paper, we discuss virtual reconstruction and interactive storytelling techniques as a research methodology and educational and presentation practices for cultural heritage sites. We present the Narrating the Past project as a case study, in order to illustrate recent changes in the preservation of social history and guided tourist trails that aim to make the visitor’s experience more than just an architectural walk through

Usability, user experience and mental workload in a mobile Augmented Reality application for digital storytelling in cultural heritage

Augmented Reality (AR) has become an increasingly used technology to support and enhance the enjoyment of cultural heritage. Particularly relevant is its importance for digital storytelling: by framing a portion of a fresco or painting with a smartphone, an AR mobile application can provide contextually relevant information, also in the form of multimedia content, that can help the user to understand the story and meaning behind the images. In this type of application, human factors are of fundamental importance for the effectiveness of the narrative: a mobile AR application must avoid distracting the user’s attention from the content in order to encourage a good level of concentration and immersion. The case study presented in this paper deals with a mobile AR application developed to guide visitors in the interpretation of the frescoes inside the Basilica of Saint Catherina of Alexandria in Galatina. The aim of the study is the analysis of the relations among usability, user experience and mental workload factors in AR-based digital storytelling

Leveraging eXtented Reality & Human-Computer Interaction for User Experi- ence in 360◦ Video

EXtended Reality systems have resurged as a medium for work and entertainment. While 360o video has been characterized as less immersive than computer-generated VR, its realism, ease of use and affordability mean it is in widespread commercial use. Based on the prevalence and potential of the 360o video format, this research is focused on improving and augmenting the user experience of watching 360o video. By leveraging knowledge from Extented Reality (XR) systems and Human-Computer Interaction (HCI), this research addresses two issues affecting user experience in 360o video: Attention Guidance and Visually Induced Motion Sickness (VIMS). This research work relies on the construction of multiple artifacts to answer the de- fined research questions: (1) IVRUX, a tool for analysis of immersive VR narrative expe- riences; (2) Cue Control, a tool for creation of spatial audio soundtracks for 360o video, as well as enabling the collection and analysis of captured metrics emerging from the user experience; and (3) VIMS mitigation pipeline, a linear sequence of modules (including optical flow and visual SLAM among others) that control parameters for visual modi- fications such as a restricted Field of View (FoV). These artifacts are accompanied by evaluation studies targeting the defined research questions. Through Cue Control, this research shows that non-diegetic music can be spatialized to act as orientation for users. A partial spatialization of music was deemed ineffective when used for orientation. Addi- tionally, our results also demonstrate that diegetic sounds are used for notification rather than orientation. Through VIMS mitigation pipeline, this research shows that dynamic restricted FoV is statistically significant in mitigating VIMS, while mantaining desired levels of Presence. Both Cue Control and the VIMS mitigation pipeline emerged from a Research through Design (RtD) approach, where the IVRUX artifact is the product of de- sign knowledge and gave direction to research. The research presented in this thesis is of interest to practitioners and researchers working on 360o video and helps delineate future directions in making 360o video a rich design space for interaction and narrative.Sistemas de Realidade EXtendida ressurgiram como um meio de comunicação para o tra- balho e entretenimento. Enquanto que o vídeo 360o tem sido caracterizado como sendo menos imersivo que a Realidade Virtual gerada por computador, o seu realismo, facili- dade de uso e acessibilidade significa que tem uso comercial generalizado. Baseado na prevalência e potencial do formato de vídeo 360o, esta pesquisa está focada em melhorar e aumentar a experiência de utilizador ao ver vídeos 360o. Impulsionado por conhecimento de sistemas de Realidade eXtendida (XR) e Interacção Humano-Computador (HCI), esta pesquisa aborda dois problemas que afetam a experiência de utilizador em vídeo 360o: Orientação de Atenção e Enjoo de Movimento Induzido Visualmente (VIMS). Este trabalho de pesquisa é apoiado na construção de múltiplos artefactos para res- ponder as perguntas de pesquisa definidas: (1) IVRUX, uma ferramenta para análise de experiências narrativas imersivas em VR; (2) Cue Control, uma ferramenta para a criação de bandas sonoras de áudio espacial, enquanto permite a recolha e análise de métricas capturadas emergentes da experiencia de utilizador; e (3) canal para a mitigação de VIMS, uma sequência linear de módulos (incluindo fluxo ótico e SLAM visual entre outros) que controla parâmetros para modificações visuais como o campo de visão restringido. Estes artefactos estão acompanhados por estudos de avaliação direcionados para às perguntas de pesquisa definidas. Através do Cue Control, esta pesquisa mostra que música não- diegética pode ser espacializada para servir como orientação para os utilizadores. Uma espacialização parcial da música foi considerada ineficaz quando usada para a orientação. Adicionalmente, os nossos resultados demonstram que sons diegéticos são usados para notificação em vez de orientação. Através do canal para a mitigação de VIMS, esta pesquisa mostra que o campo de visão restrito e dinâmico é estatisticamente significante ao mitigar VIMS, enquanto mantem níveis desejados de Presença. Ambos Cue Control e o canal para a mitigação de VIMS emergiram de uma abordagem de Pesquisa através do Design (RtD), onde o artefacto IVRUX é o produto de conhecimento de design e deu direcção à pesquisa. A pesquisa apresentada nesta tese é de interesse para profissionais e investigadores tra- balhando em vídeo 360o e ajuda a delinear futuras direções em tornar o vídeo 360o um espaço de design rico para a interação e narrativa

A user experience analysis for a mobile Mixed Reality application for cultural heritage

Mixed Reality has emerged as a valuable tool for the promotion of cultural heritage. In this context, in particular, the metaphor of virtual portals allows the virtual visit of monuments that are inaccessible or no longer exist in their original form, integrating them into the real environment. This paper presents the development of a Mixed Reality mobile application that proposes a virtual reconstruction of the church of Sant’Elia in Ruggiano, in the southern province of Lecce (Italy). By placing the virtual portal in the same place where the entrance of the church was located, the user can cross this threshold to enter inside and make a virtual journey into the past. The user experience was evaluated by administering a questionnaire to 60 users who tried the application. From the data collected, four user experience factors were identified (interest, focus of attention, presence and usability), which were compared between young and old, male and female users, and between users who had already visited the church in person and all other users. In general, the scores reveal a total independence of the other three factors from usability and a very high level of interest

Mobile heritage practices. Implications for scholarly research, user experience design, and evaluation methods using mobile apps.

Mobile heritage apps have become one of the most popular means for audience engagement and curation of museum collections and heritage contexts. This raises practical and ethical questions for both researchers and practitioners, such as: what kind of audience engagement can be built using mobile apps? what are the current approaches? how can audience engagement with these experience be evaluated? how can those experiences be made more resilient, and in turn sustainable? In this thesis I explore experience design scholarships together with personal professional insights to analyse digital heritage practices with a view to accelerating thinking about and critique of mobile apps in particular. As a result, the chapters that follow here look at the evolution of digital heritage practices, examining the cultural, societal, and technological contexts in which mobile heritage apps are developed by the creative media industry, the academic institutions, and how these forces are shaping the user experience design methods. Drawing from studies in digital (critical) heritage, Human-Computer Interaction (HCI), and design thinking, this thesis provides a critical analysis of the development and use of mobile practices for the heritage. Furthermore, through an empirical and embedded approach to research, the thesis also presents auto-ethnographic case studies in order to show evidence that mobile experiences conceptualised by more organic design approaches, can result in more resilient and sustainable heritage practices. By doing so, this thesis encourages a renewed understanding of the pivotal role of these practices in the broader sociocultural, political and environmental changes.AHRC REAC

Analysis of the narrative communication characteristics of virtual reality experiences: meaning-making components of the immersive story

Virtual reality is a technology and media that has evolved dramatically in the last decades. Undoubtedly, the medium has developed its own dynamics and narrative characteristics, due to the possibility of interaction and the ability to allow the viewer/user to focus on different levels of action. In this research, the relevant narrative characteristics in virtual reality are described based on a literature review. Secondly, a sample of online experiences of 360º virtual reality, or cinematic virtual reality (CVR), are analyzed to determine the characters and possibilities of narrative features presented. This analysis can help establish parameters and guidelines for the creation of virtual reality and 360º immersive contents in heterogeneous audiovisual and multimedia fields. The results show both the narrative and aesthetic possibilities of the analyzed videos and their technical and expressive possibilities, in terms of the ability to integrate narrative structures, as well as content in the use of innovative formal resources. In this sense, 360º immersive video becomes an added value of considerable dimensions

Immersive Journalism as Storytelling

"This book sets out cutting-edge new research and examines future prospects on 360-degree video, virtual reality (VR), and augmented reality (AR) in journalism, analyzing and discussing virtual world experiments from a range of perspectives. Featuring contributions from a diverse range of scholars, Immersive Journalism as Storytelling highlights both the opportunities and the challenges presented by this form of storytelling. The book discusses how immersive journalism has the potential to reach new audiences, change the way stories are told, and provide more interactivity within the news industry. Aside from generating deeper emotional reactions and global perspectives, the book demonstrates how it can also diversify and upskill the news industry. Further contributions address the challenges, examining how immersive storytelling calls for reassessing issues of journalism ethics and truthfulness, transparency, privacy, manipulation, and surveillance, and questioning what it means to cover reality when a story is told in virtual reality. Chapters are grounded in empirical data such as content analyses and expert interviews, alongside insightful case studies that discuss Euronews, Nonny de la Peña’s Project Syria, and The New York Times’ NYTVR application. This book is written for journalism teachers, educators, and students, as well as scholars, politicians, lawmakers, and citizens with an interest in emerging technologies for media practice.

The power of immersive technologies: a sociopsychological analysis of the relationship between immersive environments, storytelling, sentiment, and the impact on user experience

This dissertation initially focused on exploring the potential of immersive technologies for the distant future. However, the emergence of the COVID-19 virus in late 2019 disrupted the world, causing a pause in many areas. Nevertheless, the butterfly effect of the pandemic spurred the development of immersive technologies, resulting in the rise of the metaverse, web3, non-fungible tokens (NFT), and avatars, which are gaining increasing popularity. The excitement for the metaverse is growing in both academia and industry, leading to new avenues of research, digital marketing, video games, tourism, and social media. This dissertation explores this rapidly emerging technological revolution and its effects on user experience (UX)