Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

The accuracy of Automated Speech Recognition (ASR) technology has improved, but it is still imperfect in many settings. Researchers who evaluate ASR performance often focus on improving the Word Error Rate (WER) metric, but WER has been found to have little correlation with human-subject performance on many applications. We propose a new captioning-focused evaluation metric that better predicts the impact of ASR recognition errors on the usability of automatically generated captions for people who are Deaf or Hard of Hearing (DHH). Through a user study with 30 DHH users, we compared our new metric with the traditional WER metric on a caption usability evaluation task. In a side-by-side comparison of pairs of ASR text output (with identical WER), the texts preferred by our new metric were preferred by DHH participants. Further, our metric had significantly higher correlation with DHH participants' subjective scores on the usability of a caption, as compared to the correlation between WER metric and participant subjective scores. This new metric could be used to select ASR systems for captioning applications, and it may be a better metric for ASR researchers to consider when optimizing ASR systems.Comment: 10 pages, 8 figures, published in ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17

Epistemological approach to translation and accessibility in the digital context: a taxonomy proposal

Las modalidades de Traducción e Interpretación Accesible (TeIA) han sido amplio objeto de investigación desde los estudios de Traducción Audiovisual (Díaz Cintas 2007a; Orero 2007; Álvarez de Morales et al., 2012). Sin embargo, su delimitación en el contexto digital se encuentra en un estado aún propedéutico (Greco 2019: 19). En el siguiente artículo realizamos una reflexión sobre la revisión bibliográfica (Jiménez Hurtado 2007; Tercedor et al., 2007a; Báez Montero 2010; Martínez 2015; Cabezas Gay 2017; Medina Reguera 2018) para delimitar su ubicación epistemológica dentro del contexto específico de estudio, definiendo las particularidades que distinguen a las modalidades de TeIA y aportando finalmente una propuesta de taxonomía específica que incluye la descripción verbal, la interpretación a códigos visuales gestuales, la transcripción y la adaptación textual. Su análisis y estudio como elementos del Diseño Universal es clave para profundizar en la implementación de soluciones de accesibilidad efectivas para los todos los usuarios.Accessible Translation and Interpreting Modalities have been extensively researched by the Audiovisual Translation Studies (Díaz Cintas 2007a; Orero 2007; Álvarez de Morales et al., 2012). However, its delimitation in the digital context is still in a propaedeutic stage (Greco 2019: 19). In the following article, we study the bibliographic review (Jiménez Hurtado 2007; Tercedor et al., 2007a; Báez Montero 2010; Martínez 2015; Cabezas Gay 2017; Medina Reguera 2018) according to the last Audiovisual Modalities in Translation and Interpreting in order to delimit its epistemological location within the specific context of study, in one hand, we will define the particularities that distinguish them and, on the other hand, we will provide a proposal for a specific taxonomy that includes the verbal description, the interpretation of visual-gestural codes and the transcription and the text adaption. Its analysis and further research, considered as elements of Universal Design, is key to deepen the implementation of effective accessibility solutions for all users

Enhancing learning accessibility through fully automatic captioning

The simple act of listening or of taking notes while attendinga lesson may represent an insuperable burden for millions ofpeople with some form of disabilities (e.g., hearing impaired,dyslexic and ESL students). In this paper, we propose anarchitecture that aims at automatically creating captions forvideo lessons by exploiting advances in speech recognitiontechnologies. Our approach couples the usage of o-the-shelf ASR (Automatic Speech Recognition) software with anovel caption alignment mechanism that smartly introducesunique audio markups into the audio stream before givingit to the ASR and transforms the plain transcript producedby the ASR into a timecoded transcript

Integration of accessibility requirements in the design of multimedia user agents interfaces

Mención Internacional en el título de doctorThe continuous increase of multimedia content in the Web, especially video content, is not accompanied by a similar increase of accessibility; there is a lack of synchronized alternatives for the content such as captions, audio description, etc. that allow anyone with or without disability to access such content. This lack of accessibility in video content access is not only due to the lack of alternatives, but also because of the fact that user agents which deliver this content do not provide the necessary means to present them. This fact leads to the noncompliance of the current regulations and legislation in terms of accessibility. This noncompliance could be due to the lack of knowledge, or because of the fact that applying these regulations from an engineering point of view is not trivial. There is a lack of authoring tools and methodological approaches which assist in the development of an accessible product in the Engineering scope as it is the case of the development of a quality user agent which includes accessibility requirements. All these facts, multimedia content’s progressive increase on the Web, accessibility barriers both in the content and in the user agent together with current regulations and legislation regarding accessibility is what has motivated the accomplishment of this Doctoral Thesis. With this Doctoral Thesis, a set of accessibility requirements that a user agent which delivers multimedia content must fulfil is provided. Besides, a workspace is provided following a methodological approach which assists in the design and development of the interface of an accessible user agent which delivers accessible multimedia content. This workspace is composed of an architecture and models following a Model-Based User Interface Development (MBUID) approach and is oriented to be used by designers with knowledge in modeling. Finally, as a support to any professional regardless of their knowledge in modeling and in accessibility, an authoring tool based on models is offered in order to create user agents with accessibility requirements.El continuo incremento del contenido multimedia en la Web, especialmente del contenido vídeo, no va acompañado de un incremento similar de accesibilidad, hay una falta de alternativas sincronizadas al contenido como subtitulado, audiodescripción, etc., que permitan acceder a cualquier persona con y sin discapacidad a dicho contenido. Esta falta de accesibilidad en el acceso al contenido vídeo no solo se debe a la ausencia de alternativas, también es debido a que los agentes de usuario que entregan dicho contenido no proporcionan los medios necesarios para presentarlas. Este hecho da lugar a que no se cumpla la normativa y la legislación vigente en materia de accesibilidad. Dicho incumplimiento, puede ser debido al desconocimiento, o a que aplicar esa normativa desde el punto de vista de la ingeniería no es trivial. Hay una falta de herramientas de autor y de enfoques metodológicos que asistan en el desarrollo de un producto accesible en el ámbito de la Ingeniería, como es el caso del desarrollo de un agente de usuario con calidad que incluya requisitos de accesibilidad. Todos estos hechos, el incremento progresivo del contenido multimedia en la Web, las barreras de accesibilidad tanto en el contenido como en el agente de usuario junto con la normativa y legislación vigente en materia de accesibilidad es lo que ha motivado la realización de esta Tesis Doctoral. Con esta Tesis Doctoral se proporciona el conjunto de requisitos de accesibilidad que debe cumplir un agente de usuario que sirva contenido multimedia accesible. Además se proporciona un espacio de trabajo siguiendo un enfoque metodológico que asista en el diseño y desarrollo de la interfaz de un agente de usuario accesible que sirve contenido multimedia accesible. Este espacio de trabajo está compuesto de una arquitectura y modelos siguiendo el enfoque de Model-Based User Interface Development (MBUID) y está orientado a ser utilizado por diseñadores con conocimientos en modelado. Por último, como recurso de ayuda a cualquier profesional, independientemente de sus conocimientos en modelado y accesibilidad, se ofrece una herramienta de autor basada en modelos para crear agentes de usuario con requisitos de accesibilidad.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: José Antonio Macías Iglesias.- Vocal: Hugo Alexandre Paredes Guede