Mobile video comparison to help Deaf people make informed choices: a South African case study with provincial data

Deaf people use sign language to communicate and use mobile video calling to communicate with one another. Mobile video utilises much more bandwidth than text and voice communication modes, resulting in higher expenditure for communication by Deaf signers. We surveyed multiple Deaf communities to explore their level of mobile phone usage as a mode of communication. The findings indicated that despite high data cost video telephony is frequently utilized resulting in revenue generation for mobile service providers at the expense of poor Deaf end users. In South Africa, unlike for text and voice calls, both users of a video communication pay for upstream and downstream data. This paper presents a test bed comparison of the data usage and cost of the three mobile video applications with the four South African mobile network operators used by the Deaf communities. The results indicate which applications perform best on which networks and at what cost. The results can help anyone working with Deaf end users to help them make informed decisions about the use, and cost, of mobile video in South Africa.Openserver/Aria Centre of Excellenc

Developing a Sign Language Video Collection via Metadata and Video Classifiers

Video sharing sites have become a central tool for the storage and dissemination of sign language content. Sign language videos have many purposes, including sharing experiences or opinions, teaching and practicing a sign language, etc. However, due to limitations of term-based search, these videos can be hard to locate. This results in a diminished value of these sites for the deaf or hard-of-hearing community. As a result, members of the community frequently engage in a push-style delivery of content, sharing direct links to sign language videos with other members of the sign language community. To address this problem, we propose the Sign Language Digital Library (SLaDL). SLaDL is composed of two main sub-systems, a crawler that collects potential videos for inclusion into the digital library corpus, and an automatic classification system that detects and identifies sign language presence in the crawled videos. These components attempt to filter out videos that do not include sign language from the collection and to organize sign language videos based on different languages. This dissertation explores individual and combined components of the classification system. The components form a cascade of multimodal classifiers aimed at achieving high accuracy when classifying potential videos while minimizing the computational effort. A web application coordinates the execution of these two subsystems and enables user interaction (browsing and searching) with the library corpus. Since the collection of the digital library is automatically curated by the cascading classifier, the number of irrelevant results is expected to be drastically lower when compared to general-purpose video sharing sites. iii Video sharing sites have become a central tool for the storage and dissemination of sign language content. Sign language videos have many purposes, including sharing experiences or opinions, teaching and practicing a sign language, etc. However, due to limitations of term-based search, these videos can be hard to locate. This results in a diminished value of these sites for the deaf or hard-of-hearing community. As a result, members of the community frequently engage in a push-style delivery of content, sharing direct links to sign language videos with other members of the sign language community. To address this problem, we propose the Sign Language Digital Library (SLaDL). SLaDL is composed of two main sub-systems, a crawler that collects potential videos for inclusion into the digital library corpus, and an automatic classification system that detects and identifies sign language presence in the crawled videos. These components attempt to filter out videos that do not include sign language from the collection and to organize sign language videos based on different languages. This dissertation explores individual and combined components of the classification system. The components form a cascade of multimodal classifiers aimed at achieving high accuracy when classifying potential videos while minimizing the computational effort. A web application coordinates the execution of these two subsystems and enables user interaction (browsing and searching) with the library corpus. Since the collection of the digital library is automatically curated by the cascading classifier, the number of irrelevant results is expected to be drastically lower when compared to general-purpose video sharing sites. The evaluation involved a series of experiments focused on specific components of the system, and on analyzing how to best configure SLaDL. In the first set of experiments, we investigated three different crawling approaches, assessing how they compared in terms of both finding a large quantity of sign language videos and expanding the variety of videos in the collection. Secondly, we evaluated the performance of different approaches to multimodal classification in terms of precision, recall, F1 score, and computational costs. Lastly, we incorporated the best multimodal approach into cascading classifiers to reduce computation while preserving accuracy. We experimented with four different cascading configurations and analyzed their performance for the detection and identification of signed content. Given our findings of each experiment, we proposed the set up for an instantiation of SLaDL

The Listening Shift: Evaluating a Communication-Strategies Training Program for Telepractice Nurses Experiencing Hearing Challenges

Workers who wish to remain employed should be supported in doing so, even if they are experiencing age-related disabilities, such as hearing loss. I aimed to better understand the strategies from which workers with hearing loss might benefit, and how they can be supported in adopting these strategies. To collect rich data, I recruited telepractice nurses who rely on listening to make critical decisions about triaging and health care recommendations. My first research question was: What strategies exist for making telephone speech more intelligible for health care providers and patients with hearing challenges? I performed a scoping review following the Joanna Briggs Institute’s protocol. I identified 11 types of strategies, many of which required cooperation from, and disclosure to, providers’ employers, co-workers, and clients. This led me to consider the public narrative workers associated themselves with when they disclosed. Thus, my second research question was: How do Canadian newspapers portray workers with hearing loss? Through a thematic analysis of newspapers articles on this topic, I found they are predominantly portrayed as striving cheerfully both towards functioning normally and towards differentiating themselves and their hearing loss as unique and positive. To further explore how a subset of adults with hearing loss strive to work with a hearing loss, I developed an online communication-strategies training program tailored to nurses with hearing challenges. I then used a multiple case study to answer the following research question: How do nurses with hearing challenges change in terms of their telephone performance and workplace wellbeing in response to participation in an online communication strategies training program? Results suggested that nurses engaged in a problem-solving process before adopting strategies, and that strategy adoption could positively contribute to their performance. Together, the findings from these studies suggest that strategies exist to enhance the performance of workers with hearing loss, but the process of adopting these strategies can be demanding. Organizations should take steps to proactively support their nurses, health-care providers, and potentially other workers with hearing loss in identifying communication strategies and adapting them to their unique context

A Computational Model Of The Intelligibility Of American Sign Language Video And Video Coding Applications

Real-time, two-way transmission of American Sign Language (ASL) video over cellular networks provides natural communication among members of the Deaf community. Bandwidth restrictions on cellular networks and limited computational power on cellular devices necessitate the use of advanced video coding techniques designed explicitly for ASL video. As a communication tool, compressed ASL video must be evaluated according to the intelligibility of the conversation, not according to conventional definitions of video quality. The intelligibility evaluation can either be performed using human subjects participating in perceptual experiments or using computational models suitable for ASL video. This dissertation addresses each of these issues in turn, presenting a computational model of the intelligibility of ASL video, which is demonstrated to be accurate with respect to true intelligibility ratings as provided by human subjects. The computational model affords the development of video compression techniques that are optimized for ASL video. Guided by linguistic principles and human perception of ASL, this dissertation presents a full-reference computational model of intelligibility for ASL (CIM-ASL) that is suitable for evaluating compressed ASL video. The CIM-ASL measures distortions only in regions relevant for ASL communication, using spatial and temporal pooling mechanisms that vary the contribution of distortions according to their relative impact on the intelligibility of the compressed video. The model is trained and evaluated using ground truth experimental data, collected in three separate perceptual studies. The CIM-ASL provides accurate estimates of subjective intelligibility and demonstrates statistically significant improvements over computational models traditionally used to estimate video quality. The CIM-ASL is incorporated into an H.264/AVC compliant video coding framework, creating a closed-loop encoding system optimized explicitly for ASL intelligibility. This intelligibility optimized coder achieves bitrate reductions between 10% and 42% without reducing intelligibility, when compared to a general purpose H.264/AVC encoder. The intelligibility optimized encoder is refined by introducing reduced complexity encoding modes, which yield a 16% improvement in encoding speed. The purpose of the intelligibility optimized encoder is to generate video that is suitable for real-time ASL communication. Ultimately, the preferences of ASL users determine the success of the intelligibility optimized coder. User preferences are explicitly evaluated in a perceptual experiment in which ASL users select between the intelligibility optimized coder and a general purpose video coder. The results of this experiment demonstrate that the preferences vary depending on the demographics of the participants and that a significant proportion of users prefer the intelligibility optimized coder

Governing Confidence: Rhetoric, Affect, and Post-Crisis Financial Education

The 2007-08 financial crisis has been characterized as a “crisis of confidence” (Akerlof & Shiller, 2009), a span of time during which the non-discursive energy needed to compel Americans towards profit-producing decisions evaporated. Amidst this decline, the US lost its competitive edge in the global marketplace. Initial responses to the crisis by national leaders failed, triggering a revision to reasoning that resulted in a new argument taken up by central government: the lack of financial knowledge experienced by the majority of US citizens led to a population of ignorant decision makers lacking the confidence needed to take the risks necessary to propel the country’s economy forward. According to the problematization, the solution necessary was the revision of the country’s financial education curriculum. Embracing this argument, both Presidents Bush and Obama called for the development of councils that would evaluate the status of financial education and use that information to recommend changes to the discourse of financial education in the future. This dissertation uses Foucault’s (1991, 2007, 2008) theory of governmentality alongside the affect scholarship of Brennan (2004) and Grusin’s (2010) work on the digital mediation of affect to examine these arguments and the technologies of governance they produced that would motivate US citizens to take control of their financial situations through actions made with confidence that would benefit these individual decision makers as well as the US economy

Comparison and evaluation of mass video notification methods used to assist Deaf people

Magister Scientiae - MScIn South Africa, Deaf people communicate with one another and the broader community by means of South African Sign Language. The majority of Deaf people who have access to a mobile phone (cell phone) use Short Message Service (SMS) to communicate and share information with hearing people, but seldom use it among themselves. It is assumed that video messaging will be more accessible to Deaf people, since their level of literacy may prevent them from making effective use of information that is disseminated via texting/SMS. The principal objective of the esearch was to explore a cost-effective and efficient mass multimedia messaging system. The intention was to adapt a successful text-based mass notification system, developed by a local nongovernmental organization (NGO), to accommodate efficient and affordable video mass messaging for Deaf people. The questions that underpin this research are: How should video- streaming mass-messaging methods be compared and evaluated to find the most suitable method to deliver an affordable and acceptable service to Deaf people? What transport vehicles should be considered: Multimedia Message Service (MMS), the web, electronic mail, or a cell phone resident push/pullapplication? Which is the most cost effective? And, finally: How does the video quality of the various transport vehicles differ in terms of the clarity of the sign language as perceived by the Deaf? The soft-systems methodology and a mixed-methods methodology were used to address the research questions. The soft-systems methodology was followed to manage the research process and the mixed-methods research methodology was followed to collect data. Data was collected by means of experiments and semi-structured interviews. A prototype for mobile phone usage was developed and evaluated with Deaf members the NGO Deaf Community of Cape Town. The technology and internet usage of the Deaf participants provided background information. The Statistical Package for Social Science (SPSS) was used to analyse the quantitative data, and content analysis was used to analyse the documents and interviews. All of the Deaf participants used their mobile phones for SMS and the majority (81.25%) used English to type messages; however, all indicated that they would have preferred to use South Africa sign language on their mobile phones if it were available. And they were quite willing to pay between 75c and 80c per message for using such a video-messaging service.Of the transport vehicles demonstrated, most Deaf people indic indicated that they preferred to use the SMS prototype (with a web link to the video) rather than the MMS prototype with the video attached. They were, however, very concerned about the cost of using the system, as well as the quality of the sign language videos.South Afric

Quality aspects of Internet telephony

Internet telephony has had a tremendous impact on how people communicate. Many now maintain contact using some form of Internet telephony. Therefore the motivation for this work has been to address the quality aspects of real-world Internet telephony for both fixed and wireless telecommunication. The focus has been on the quality aspects of voice communication, since poor quality leads often to user dissatisfaction. The scope of the work has been broad in order to address the main factors within IP-based voice communication. The first four chapters of this dissertation constitute the background material. The first chapter outlines where Internet telephony is deployed today. It also motivates the topics and techniques used in this research. The second chapter provides the background on Internet telephony including signalling, speech coding and voice Internetworking. The third chapter focuses solely on quality measures for packetised voice systems and finally the fourth chapter is devoted to the history of voice research. The appendix of this dissertation constitutes the research contributions. It includes an examination of the access network, focusing on how calls are multiplexed in wired and wireless systems. Subsequently in the wireless case, we consider how to handover calls from 802.11 networks to the cellular infrastructure. We then consider the Internet backbone where most of our work is devoted to measurements specifically for Internet telephony. The applications of these measurements have been estimating telephony arrival processes, measuring call quality, and quantifying the trend in Internet telephony quality over several years. We also consider the end systems, since they are responsible for reconstructing a voice stream given loss and delay constraints. Finally we estimate voice quality using the ITU proposal PESQ and the packet loss process. The main contribution of this work is a systematic examination of Internet telephony. We describe several methods to enable adaptable solutions for maintaining consistent voice quality. We have also found that relatively small technical changes can lead to substantial user quality improvements. A second contribution of this work is a suite of software tools designed to ascertain voice quality in IP networks. Some of these tools are in use within commercial systems today

Developing a Sign Language Video Collection via Metadata and Video Classifiers

Video sharing sites have become a central tool for the storage and dissemination of sign language content. Sign language videos have many purposes, including sharing experiences or opinions, teaching and practicing a sign language, etc. However, due to limitations of term-based search, these videos can be hard to locate. This results in a diminished value of these sites for the deaf or hard-of-hearing community. As a result, members of the community frequently engage in a push-style delivery of content, sharing direct links to sign language videos with other members of the sign language community. To address this problem, we propose the Sign Language Digital Library (SLaDL). SLaDL is composed of two main sub-systems, a crawler that collects potential videos for inclusion into the digital library corpus, and an automatic classification system that detects and identifies sign language presence in the crawled videos. These components attempt to filter out videos that do not include sign language from the collection and to organize sign language videos based on different languages. This dissertation explores individual and combined components of the classification system. The components form a cascade of multimodal classifiers aimed at achieving high accuracy when classifying potential videos while minimizing the computational effort. A web application coordinates the execution of these two subsystems and enables user interaction (browsing and searching) with the library corpus. Since the collection of the digital library is automatically curated by the cascading classifier, the number of irrelevant results is expected to be drastically lower when compared to general-purpose video sharing sites. iii Video sharing sites have become a central tool for the storage and dissemination of sign language content. Sign language videos have many purposes, including sharing experiences or opinions, teaching and practicing a sign language, etc. However, due to limitations of term-based search, these videos can be hard to locate. This results in a diminished value of these sites for the deaf or hard-of-hearing community. As a result, members of the community frequently engage in a push-style delivery of content, sharing direct links to sign language videos with other members of the sign language community. To address this problem, we propose the Sign Language Digital Library (SLaDL). SLaDL is composed of two main sub-systems, a crawler that collects potential videos for inclusion into the digital library corpus, and an automatic classification system that detects and identifies sign language presence in the crawled videos. These components attempt to filter out videos that do not include sign language from the collection and to organize sign language videos based on different languages. This dissertation explores individual and combined components of the classification system. The components form a cascade of multimodal classifiers aimed at achieving high accuracy when classifying potential videos while minimizing the computational effort. A web application coordinates the execution of these two subsystems and enables user interaction (browsing and searching) with the library corpus. Since the collection of the digital library is automatically curated by the cascading classifier, the number of irrelevant results is expected to be drastically lower when compared to general-purpose video sharing sites. The evaluation involved a series of experiments focused on specific components of the system, and on analyzing how to best configure SLaDL. In the first set of experiments, we investigated three different crawling approaches, assessing how they compared in terms of both finding a large quantity of sign language videos and expanding the variety of videos in the collection. Secondly, we evaluated the performance of different approaches to multimodal classification in terms of precision, recall, F1 score, and computational costs. Lastly, we incorporated the best multimodal approach into cascading classifiers to reduce computation while preserving accuracy. We experimented with four different cascading configurations and analyzed their performance for the detection and identification of signed content. Given our findings of each experiment, we proposed the set up for an instantiation of SLaDL

Intelligent Mobile Learning Interaction System (IMLIS): A Personalized Learning System for People with Mental Disabilities

The domain of learning context for people with special needs is a big challenge for digi- tal media in education. This thesis describes the main ideas and the architecture of a system called Intelligent Mobile Learning Interaction System (IMLIS) that provides a mobile learning environment for people with mental disabilities. The design of IMLIS aims to enhance personalization aspects by using a decision engine, which makes deci- sions based on the user s abilities, learning history and reactions to processes. It allows for adaptation, adjustment and personalization of content, learning activities, and the user interface on different levels in a context where learners and teachers are targeting autonomous learning by personalized lessons and feedback. Due to IMLIS dynamic structure and flexible patterns, it is able to meet the specific needs of individuals and to engage them in learning activities with new learning motivations. In addition to support- ing learning material and educational aspects, mobile learning fosters learning across context and provides more social communication and collaboration for its users. The suggested methodology defines a comprehensive learning process for the mentally disabled to support them in formal and informal learning. We apply knowledge from the field of research and practice to people with mental disabilities, as well as discuss the pedagogical and didactical aspects of the design