Inclusive Communication with Augmented Reality for deaf and hard of hearing

UIDB/05021/2020 UIDP/05021/2020Deafness is an often undervalued but increasing problem amongst world’s population. Besides the difficulty in hearing sounds, it involves many cognitive and emotional issues, like learning difficulties, isolation and disempowerment. In this paper, designing artefacts for deaf people is analyzed from two different perspectives: technology and communication science. Regarding technical issues, research shows how Augmented Reality can be applied using screen interfaces or smart glasses translating sounds into visual stimuli; from the communication side, the focus is on storytelling and how it can be combined with the technology referred to engage people, enhance learning activity and create a community. By looking into these two aspects, the suggested approach is to merge them in a conceptual creative project that can be appealing and useful to the public, through the use of interactive storytelling, while also using the visual benefits of an immersive Augmented Reality experience.publishersversionpublishe

Toward A Theory of Media Reconciliation: An Exploratory Study of Closed Captioning

This project is an interdisciplinary empirical study that explores the emotional experiences resulting from the use of the assistive technology closed captioning. More specifically, this study focuses on documenting the user experiences of both the D/deaf and Hearing multimedia user in an effort to better identify and understand those variables and processes that are involved with facilitating and supporting connotative and emotional meaning making. There is an ever present gap that defines closed captioning studies thus far, and this gap is defined by the emphasis on understanding and measuring denotative meaning making behavior while largely ignoring connotative meaning making behavior that is necessarily an equal participant in a user\u27s viewing experience. This study explores connotative and emotional meaning making behaviors so as to better understand the behavior exhibited by users engaged with captioned multimedia. To that end, a mixed methods design was developed that utilizes qualitative methods from the field of User Experience (UX) to explore connotative equivalence between D/deaf and Hearing users and an augmented version of S. R. Gulliver and G. Ghinea\u27s (2003) quantitative measure Information Assimilation (IA) from the field of Human Computer Interaction (HCI) to measure the denotative equivalence between the two user types. To measure denotative equivalence a quiz containing open-ended questions to measure IA was used. To measure connotative equivalence the following measures were used: 1) Likert scales to measure users\u27 confidence in answers to open-ended questions. 2) Likert scale to measure a users\u27 interest in the stimulus. 3) Open - ended questions to identify scenes that elicited the strongest emotional responses from users. 4) Four- level response questions with accompanying Likert scales to determine strength of emotional reaction to three select excerpts from the stimulus. 5) An interview consisting of three open- ended questions and one fixed - choice question. This study found that there were no major differences in the denotative equivalence between the D/deaf and Hearing groups; however, there were important differences in the emotional reactions to the stimulus that indicate there was not connotative equivalence between the groups in response to the emotional content. More importantly, this study found that the strategies used to understand the information users were presented with in order to create both denotative and connotative meaning differed between groups and individuals within groups. To explain such behaviors observed, this work offers a theory of Media Reconciliation based on Wolfgang Iser\u27s (1980) phenomenological theory about the \u27virtual text\u27

e-Archeo. A pilot national project to valorize Italian archaeological parks through digital and virtual reality technologies

Commissioned to ALES spa by the Ministry of Culture (MiC), the e-Archeo project was born with the intention of enhancing and promoting knowledge of some Italian archaeological sites with a considerable narrative potential that has not yet been fully expressed. The main principle that guided the choice of the sites and the contents was of illustrating the various cultures and types of settlements present in the Italian territory. Eight sites were chosen, spread across the national territory from north to south, founded by Etruscans, Greeks, Phoenicians, natives and Romans. e-Archeo has developed multimedia, integrated and multi-channel solutions for various uses and types of audiences, adopting both scientific and narrative and emotional languages. Particular attention was paid to multimedia accessibility, technological sustainability and open science. The e-Archeo project was born from a strong synergy between public entities, research bodies and private industries thanks to the collaboration of MiC and ALES with the CNR ISPC, 10 Italian Universities, 12 Creative Industries and the Italian National Television (RAI). This exceptional and unusual condition made it possible to realise all the project’s high-quality contents and several outputs in only one and a half years

A sound idea: An investigation into accessible video game design for the deaf and hard of hearing

A widely accepted, and incorrect, assumption towards hearing accessibility in video games is that deaf and hard of hearing (DHH) users are those who encounter the least barriers and are generally well catered for. Rapid advancement in video game technology has seen video game sound evolve from simple blips generated by internal circuitry to fully realised digital audio used to convey critical information. To accommodate the DHH, this information needs to be conveyed in an alternative manner. However, evidence suggests existing accessible design solutions for the DHH lack specificity and are insufficient. Thus, the inability to hear, or hear well, has historically resulted in DHH users left with impeded experience and gameplay. This thesis describes an investigation to address the primary research question: How might accessible video game design practices be facilitated to better accommodate the deaf and hard of hearing? To examine this question, an action research method as part of a transformative mixed methods methodology was used. Data collection procedures included critical analysis of literature, observations, and a cross-sectional self-administered survey for triangulation. The critical analysis of literature exposed issues relating to accessible video game design, particularly in relation to the identification of solutions and technical implementation. Further, issues related to the classification of video game software were identified. This posed potential problems with identification of game design methods and led to the development of a new video game classification model. The new model informed an analysis on the methods used for the design of video games, and results were visually represented and mapped to the different approaches to accessible design. Subsequent analysis determined that a game assessment framework is a suitable approach to facilitating accessible design. Further investigation identified visual feedback as the most suitable form of complementary feedback to game audio. This led to the development of a new model to classify visual feedback elements used in video games, and identification of audio feedback categories based on diegetic film theory. Through triangulation of results, a new game feedback model (GFM) was developed. The GFM was used for observational experimentation to identify and classify individual visual feedback elements used in video games. Each element was analysed and mapped to categories of game sound. The resulting model, with populated data, was used to determine what visual feedback elements may be used to complement specific categories of critical game audio. A survey was subsequently used for triangulation, and resulted in amendments to the final model. Through iterative development, and interpretation of findings, the research culminated in the development of a game assessment framework. The three-step framework aids in the classification of game sounds; assesses the impact of those game sounds; and provides recommendations for complementary visual feedback elements for sounds identified as having an adverse impact on user experience and gameplay if they were to be removed. The framework is innovative and has the potential to provide practical guidance for developers of video games. In addition, this research provides the foundation for future research, with the potential to influence accessible game design for the DHH

Accessibility at Film Festivals: Guidelines for Inclusive Subtitling

In today's media-dominated world, the imperative for accessibility has never been greater, and ensuring that audiovisual experiences cater to individuals with sensory disabilities has become a pressing concern. One of the key initiatives in this endeavour is inclusive subtitling (IS), a practice rooted in the broader contexts of subtitling for the deaf and hard of hearing (SDH/CC), audiovisual translation studies (AVTS), media accessibility studies (MAS), and the evolving field of Deaf studies (DS). This study aims to offer a comprehensive exploration of how inclusive subtitling contributes to fostering accessible and inclusive audiovisual experiences, with a particular focus on its implications within the unique environment of film festivals. To gain a holistic perspective of inclusive subtitling, it is essential to examine its lineage in relation to analogous practices, which is the focus of the first chapter. Inclusive subtitling is an extension of SDH/CC, designed for individuals with hearing impairments, and SDH/CC, in turn, is a nuanced variation of traditional subtitling extensively explored within the realm of AVTS. To encapsulate the diverse techniques and modalities aimed at making audiovisual content universally accessible, the study recognises the term "Audiovisual Accessibility" (AVA). The second chapter explores the interconnection of accessibility studies (AS), AVTS, and MAS, highlighting their symbiotic relationship and their role in framing inclusive subtitles within these fields. These interconnections are pivotal in shaping a framework for the practice of inclusive subtitling, enabling a comprehensive examination of its applicability and research implications. The third chapter delves into Deaf studies and the evolution of Deafhood, which hinges on the history and culture of Deaf individuals. This chapter elucidates the distinction between ‘deafness’ as a medical construct and ‘Deafhood’ as a cultural identity, crucial to the understanding of audiovisual accessibility and its intersection with the Deaf community's perspectives. In the fourth chapter, the focus turns to the exploration of film festivals, with a specific emphasis on the crucial role of subtitles in enhancing accessibility, particularly when films are presented in their original languages. The chapter marks a critical point, highlighting the inherent connection between subtitles and the immersive nature of film festivals that aspire to promote inclusivity in the cinematic experience. The emphasis on inclusivity extends to the evolution of film festivals, giving rise to more advanced forms, including accessible film festivals and Deaf film festivals. At the core of the chapter is a thorough examination of the corpus, specifically, the SDH/CC of films spanning the editions from 2020 to 2023 of two highly significant film festivals, namely BFI Flare and the London Film Festival. The corpus serves as the foundation upon which my research unfolds, providing a nuanced understanding of the role subtitles play in film festival contexts. The main chapter, chapter five, thoroughly analyses the technical and linguistic aspects of inclusive subtitling, drawing insights from the Inclusive Subtitling Guidelines - a two version document devised by myself - and offering real-world applications supported by a case study at an Italian film festival and another case study of the short film Pure, with the relevant inclusive subtitles file annexed. In conclusion, the research sets the stage for a comprehensive exploration of inclusive subtitling's role in ensuring accessible and inclusive audiovisual experiences, particularly within film festivals. It underscores the importance of accessibility in the world of audiovisual media and highlights the need for inclusive practices to cater to diverse audiences

City Tells:

City Tells. Guidelines to an Emotional Wayfinding System were developed to provide wayfinding information to visitors walking through historic environments and to ensure that unknown urban places become more welcoming, easier to navigate and more enjoyable for both visitors and tourists

XR Academia:Research and Experiences in Virtual Reality, Augmented Reality, Mixed Reality, and Artificial Intelligence in Latin America and Europe

The book XR Academia: Research and Experiences in Virtual Reality, Augmented Reality, Mixed Reality, and Artificial Intelligence in Latin America and Europe, has at its core the objective of making immersive technology accessible and visible worldwide, with the simultaneous breaking-down of linguistic barriers. Both European and Latin American authors can read each other’s work(s), allowing knowledge and experience in extended reality to be shared. Another important aspect of XR Academia is its attempt to introduce an open science contribution to the issues of immersive technologies, in order to inspire new generations that do not have access to increasingly expensive publications. This volume includes fourteen selected chapters from presenters from the 2020 and 2021 events. These chapters describe research and experiences on a wide range of XR applications, which include entertainment, health, narration, education, psychotherapy, guidance, language, culture and arts. Considering that great inventions and innovations are developed in Latin America but fail to be published internationally, our aim was to open a door to allow the permanent exchange between two languages: Spanish and English

A face in the crowd: imagining individual and collective disabled identities in contemporary China

This essay explores the concept of a “disabled crowd” in the Chinese cultural and social imagination. It highlights key terms, analogies, and cultural locations of disability that are seen to relate to and inform group identity and collective behavior, and interrogates these through close readings of a range of contemporary personal narratives and other sources related to the sociopolitical context. It reveals that the appropriation of new and enhanced opportunities for self-representation and self-advocacy is enabling a wide range of disabled people, both on an individual and group level, to “speak out” about their experiences of disability and, just as importantly, “be heard”. What it also shows, however, is that while some disabled people identify with and appear to derive intense personal and social benefit from being associated with a “disabled crowd,” others have used these new opportunities to re-imagine or distance themselves from that same crowd by offering up alternative narratives of what it means to be disabled in China today. In doing so, therefore, the article demonstrates the closely interrelated nature of self and group empowerment and identity in a country where the state has attempted to act as the guardian and voice of disabled people since the 1980s, but where that influence has been increasingly disrupted by voices from across the spectrum of disability

A face in the crowd: imagining individual and collective disabled identities in contemporary China

This essay explores the concept of a “disabled crowd” in the Chinese cultural and social imagination. It highlights key terms, analogies, and cultural locations of disability that are seen to relate to and inform group identity and collective behavior, and interrogates these through close readings of a range of contemporary personal narratives and other sources related to the sociopolitical context. It reveals that the appropriation of new and enhanced opportunities for self-representation and self-advocacy is enabling a wide range of disabled people, both on an individual and group level, to “speak out” about their experiences of disability and, just as importantly, “be heard”. What it also shows, however, is that while some disabled people identify with and appear to derive intense personal and social benefit from being associated with a “disabled crowd,” others have used these new opportunities to re-imagine or distance themselves from that same crowd by offering up alternative narratives of what it means to be disabled in China today. In doing so, therefore, the article demonstrates the closely interrelated nature of self and group empowerment and identity in a country where the state has attempted to act as the guardian and voice of disabled people since the 1980s, but where that influence has been increasingly disrupted by voices from across the spectrum of disability