AR Comic Chat

Live speech transcription and captioning are important for the accessibility of deaf and hard of hearing individuals, especially in situations with no visible ASL translators. If live captioning is available at all, it is typically rendered in the style of closed captions on a display such as a phone screen or TV and away from the real conversation. This can potentially divide the focus of the viewer and detract from the experience. This paper proposes an investigation into an alternative, Augmented Reality driven approach to the display of these captions, using deep neural networks to compute, track and associate deep visual and speech descriptors in order to maintain captions as speech bubbles above the speaker

Subtitling for the Deaf and the Hard-of-hearing: A Reception Study in the Turkish Context

This study aims to contribute to a better understanding of subtitling for people with hearing impairments and to improve the accessibility to audiovisual material for hearing-impaired viewers in Turkey. It starts by providing a detailed general overview of the current state of accessibility and includes a detailed discussion on existing legislation, an outline of the limited practice of subtitling for the deaf and the hard-of-hearing (SDH) in Turkish and a profile of the assumed target audience. The ultimate goal of this research is to create a set of guidelines that can be used in the production of quality SDH in Turkey. In order to achieve these aims, the study adopts a product-oriented descriptive approach and first investigates the guidelines applied in countries where SDH has long been established as a professional practice in an attempt to reveal some of the shared values of good practice as well as potential divergences. Following this descriptive analysis, some of the key contradicting practices in the guidelines – speaker identification, reading speed, indication of sound and paralinguistic information – are tested on an audience of (37) Turkish hearing-impaired viewers so as to unveil their needs and preferences within the framework of Audience Reception Theory. Quantitative data on the preferences of Turkish viewers was collected by means of questionnaires filled in by the participants after they had watched different sets of subtitles, each of them testing a different feature. Further qualitative data was obtained through interviews conducted with four participants who took part in the experiment so as to generate more in-depth information regarding their preferences. The results yielded by the statistical analysis of the quantitative data and the interpretive phenomenological analysis of the qualitative data culminated in the drafting of a set of guidelines that can be used in the production of SDH in Turkey

Using Avatars for Improving Speaker Identification in Captioning

Abstract. Captioning is the main method for accessing television and film content by people who are deaf or hard-of-hearing. One major difficulty consistently identified by the community is that of knowing who is speaking particularly for an off screen narrator. A captioning system was created using a participatory design method to improve speaker identification. The final prototype contained avatars and a coloured border for identifying specific speakers. Evaluation results were very positive; however participants also wanted to customize various components such as caption and avatar location

Speaker-following Video Subtitles

We propose a new method for improving the presentation of subtitles in video (e.g. TV and movies). With conventional subtitles, the viewer has to constantly look away from the main viewing area to read the subtitles at the bottom of the screen, which disrupts the viewing experience and causes unnecessary eyestrain. Our method places on-screen subtitles next to the respective speakers to allow the viewer to follow the visual content while simultaneously reading the subtitles. We use novel identification algorithms to detect the speakers based on audio and visual information. Then the placement of the subtitles is determined using global optimization. A comprehensive usability study indicated that our subtitle placement method outperformed both conventional fixed-position subtitling and another previous dynamic subtitling method in terms of enhancing the overall viewing experience and reducing eyestrain

Accessibility to Theater for Deaf and Deaf-blind People: Legal, Language and Artistic Considerations

Without accessibility, theater can be meaningless to the deaf, hard of hearing, and deaf-blind consumers. As part of a larger study conducted by B. Kilpatrick (2007), the authors interviewed 38 participants who have been professionally involved in deaf children’s theater as to their opinions related to theater accessibility options. Their responses bring forward for discussion options ranging from English text-based accessibility, the closest to the English language, to shadow interpreting, which provides accessibility closest to the play being delivered in full in American Sign Language. Using historical research methods, semi-structured and structured interviews, open-ended questions, archival materials, and published documents on theater interpreting, the authors provide a descriptive commentary about accessibility options based on legal, language and artistic considerations. Following these descriptions, the authors recommend that interpreter training programs include theater interpretation techniques

Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences

Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences is a comprehensive guide to the theory and practice of captioning and subtitling, a discipline that has evolved quickly in recent years. This guide is of a practical nature and contains examples and exercises at the end of each chapter. Some of the tasks stimulate reflection on the practice and reception, while others focus on particular captioning and SDH areas, such as paralinguistic features, music and sound effects.The requirements of d/Deaf and hard of hearing audiences are analysed in detail and are accompanied by linguistic and technical considerations. These considerations, though shared with generic subtitling parameters, are discussed specifically with d/Deaf and hard of hearing audiences in mind. The reader will become familiar with the characteristics and needs of d/Deaf and hard of hearing audiences, and the diversity – including cultural and linguistic differences – within this group of people. Based on first-hand experience in the field, the book also provides a step-by-step guide to making live performances accessible to d/Deaf and hard of hearing audiences. As well as exploring all linguistic and technical matters related to the creation of captions, aspects related to the overall set up of the captioned performance are discussed. The guide will be valuable reading to students of audiovisual translation at undergraduate and postgraduate level, to professional subtitlers and captioners, and to any organisation or venue that engages with d/Deaf and hard of hearing people. ; Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences is a comprehensive guide to the theory and practice of captioning and subtitling

Toward A Theory of Media Reconciliation: An Exploratory Study of Closed Captioning

This project is an interdisciplinary empirical study that explores the emotional experiences resulting from the use of the assistive technology closed captioning. More specifically, this study focuses on documenting the user experiences of both the D/deaf and Hearing multimedia user in an effort to better identify and understand those variables and processes that are involved with facilitating and supporting connotative and emotional meaning making. There is an ever present gap that defines closed captioning studies thus far, and this gap is defined by the emphasis on understanding and measuring denotative meaning making behavior while largely ignoring connotative meaning making behavior that is necessarily an equal participant in a user\u27s viewing experience. This study explores connotative and emotional meaning making behaviors so as to better understand the behavior exhibited by users engaged with captioned multimedia. To that end, a mixed methods design was developed that utilizes qualitative methods from the field of User Experience (UX) to explore connotative equivalence between D/deaf and Hearing users and an augmented version of S. R. Gulliver and G. Ghinea\u27s (2003) quantitative measure Information Assimilation (IA) from the field of Human Computer Interaction (HCI) to measure the denotative equivalence between the two user types. To measure denotative equivalence a quiz containing open-ended questions to measure IA was used. To measure connotative equivalence the following measures were used: 1) Likert scales to measure users\u27 confidence in answers to open-ended questions. 2) Likert scale to measure a users\u27 interest in the stimulus. 3) Open - ended questions to identify scenes that elicited the strongest emotional responses from users. 4) Four- level response questions with accompanying Likert scales to determine strength of emotional reaction to three select excerpts from the stimulus. 5) An interview consisting of three open- ended questions and one fixed - choice question. This study found that there were no major differences in the denotative equivalence between the D/deaf and Hearing groups; however, there were important differences in the emotional reactions to the stimulus that indicate there was not connotative equivalence between the groups in response to the emotional content. More importantly, this study found that the strategies used to understand the information users were presented with in order to create both denotative and connotative meaning differed between groups and individuals within groups. To explain such behaviors observed, this work offers a theory of Media Reconciliation based on Wolfgang Iser\u27s (1980) phenomenological theory about the \u27virtual text\u27

Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences

Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences is a comprehensive guide to the theory and practice of captioning and subtitling, a discipline that has evolved quickly in recent years. This guide is of a practical nature and contains examples and exercises at the end of each chapter. Some of the tasks stimulate reflection on the practice and reception, while others focus on particular captioning and SDH areas, such as paralinguistic features, music and sound effects.The requirements of d/Deaf and hard of hearing audiences are analysed in detail and are accompanied by linguistic and technical considerations. These considerations, though shared with generic subtitling parameters, are discussed specifically with d/Deaf and hard of hearing audiences in mind. The reader will become familiar with the characteristics and needs of d/Deaf and hard of hearing audiences, and the diversity – including cultural and linguistic differences – within this group of people. Based on first-hand experience in the field, the book also provides a step-by-step guide to making live performances accessible to d/Deaf and hard of hearing audiences. As well as exploring all linguistic and technical matters related to the creation of captions, aspects related to the overall set up of the captioned performance are discussed. The guide will be valuable reading to students of audiovisual translation at undergraduate and postgraduate level, to professional subtitlers and captioners, and to any organisation or venue that engages with d/Deaf and hard of hearing people. ; Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences is a comprehensive guide to the theory and practice of captioning and subtitling