Telecommunications bridging between deaf and hearing users in South Africa

This paper presents a description of telecommunications for Deaf people in South Africa by contrasting the situation in the developed world. We explain the main motivations for providing a locally appropriate solution. Research activity began with a community-based field trial of TTYs. The recommendations of the trial motivated an innovative approach to provide synchronous and accessible relay services. Drawing from international technological developments, we designed a software solution called the SoftBridge. We employ iterative Action Research cycles to drive technological modifications to the SoftBridge and to encourage adoption by the end-user community.Telkom, Siemens, Cisco, THRIP, SANPA

Web-based telephony bridges for the deaf

A Teldem provides text-based telephony services to the deaf. Despite an array of text-based communications mechanisms on the web, the Teldem is the only means of synchronous telecommunication available to a deaf person. We examine a human-assisted relay that provides a bridge between the deaf and the hearing, and then propose several additional bridges. These bridges take advantage of both standard and cutting edge technologies. We present each bridge's impact on the deaf with respect to synchrony, connectivity, independence, and the Digital Divide. It is hoped that by deploying trials for each bridge, we can eventually provide the deaf with equal access.Telkom, Siemens, THRIPDepartment of HE and Training approved lis

Towards communication and information access for deaf people

In tightly circumscribed communication situations an interactive system resident on a mobile device can assist Deaf people with their communication and information needs. The Deaf users considered here use South African Sign Language and information is conveyed by a collection of pre-recorded video clips and images. The system was developed according to our method of community-based co-design. We present several stages of the development as a series of case studies and highlight our experience. The first stage involved ethnographically inspired methods such as cultural probes. In the next stage we co-designed a medical consultation system that was ultimately dropped for technical reasons. A smaller system was developed for pharmaceutical dispensing and successfully implemented and tested. It now awaits deployment in an actual pharmacy. We also developed a preliminary authoring tool to tackle the problem of content generation for interactive computer literacy training. We are also working on another medical health information tool. We intend that a generic authoring tool be able to generate mobile applications for all of these scenarios. These mobile applications bridge communication gaps for Deaf people via accessible and affordable assistive technology

Preparation of Deaf end-users and the softbridge for semi-automated relay trials

Following on the development of several prototypes, we have built a semi-automated Deaf Telephony prototype on the SoftBridge platform. This prototype relays text and speech between Deaf users on the Internet and hearing users on the telephone system. Previous work with a pilot trial in the laboratory revealed several opportunities for enhancement. We added a Wizard of Oz (WoOz) to replace the poorly performing automatic speech recognition functionality as well as H.323 breakout, more extensive logging and advanced call initiation functionality. In order to trial the current prototype, we initiated an Information and Communication Technology (ICT) training programme with the Deaf Community of Cape Town. Twenty Deaf users participated in the training. In addition to the training, much baseline user data was collected to give an indication of how Deaf users communicate with hearing users as well as how familiar they are with ICT devices and services. The work for the rest of this year requires us to recruit and train a WoOz operator. Subsequent trials will essentially consist of monthly cycles of prototype introduction, training, automated metric and log collection, user feedback and then feature enhancement. Linguistic analyses of the text output of the Deaf users will be analyzed. We hope to refine the SoftBridge prototype to fit the needs of the Deaf and hearing users, from both technical and social viewpoints. We expect that these iterative cycles will continue for some time and will teach us many lessons regarding multi-modal semi-synchronous IP-based communications systems.Telkom, Siemens, THRIP, SANPA

Telgo323: an H.323 Bridge for deaf telephony

We have developed a prototype bridge that relays text and speech between Teldem, a text telephone for the Deaf, and a standard telephone or H.323 endpoint. Telgo323 uses modified H.323 media gateways and open source Text to Speech and Speech to Text software. The approach allows for easy integration of new tools as the technologies mature. This paper presents the design of the implementation prototype, discusses Teldem tone decoding, and suggests directions for future work. The Telgo323 provides evidence that an automated relay bridge is imminently viable for the Deaf Community, and further demonstrates an attractive approach for building bridges over the Digital Divide.Telkom, Siemens, THRIPDepartment of HE and Training approved lis

Usability of an authoring tool for generalised scenario creation for signsupport

This paper presents the usability testing results for an authoring tool that generalises scenario creation for a tool called SignSupport. SignSupport is a mobile communication tool for Deaf people that currently runs on an Android smartphone. The authoring tool is computer-based software that helps a domain expert, with little or no programming skills, design and populate a limited domain conversation scenario between a Deaf person and a hearing person, e.g., when a Deaf patient collects medication at a hospital pharmacy or when a Deaf learner is taking a computer literacy course. SignSupport provides instructions to the Deaf person in signed language videos on a mobile device. The authoring tool enables the creation and population of such scenarios on a computer for subsequent 'playback' on a mobile device. The output of this authoring tool is an XML script, alongside a repository of media files that can be used to render the SignSupport mobile app on any platform. Our concern now is to iteratively develop the user interface for the authoring tool, focusing on the domain experts who create the overall flow and content for a given scenario. The current authoring tool was evaluated for usability; for both pharmacy and ICDL course scenarios with purposive sampling. The findings suggest that the authoring tool can generalise SignSupport for multiple limited domain scenarios, mobile platforms and signed languages.Telkom, Cisco, Aria Technologies, THRIPDepartment of HE and Training approved lis

Teaching design for development in computer science

When undertaking a project in community-based co-design, students and their teachers must embrace uncertainty

SoftBridge in action: the first Deaf telephony pilot

Following on several partial prototypes, we built an automated Deaf Telephony bridging application with the SoftBridge platform. The SoftBridge performs multi-modal bridging in real-time using Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) utilities accessed via web services. To perform Action Research with the Deaf community in Cape Town, we invited a Deaf user (DU) to participate in a pilot study. We conducted three tests varying the input/output modalities of the hearing user (HU). The DU had a standard text in/text out Instant Messaging client. The HU client used the following specific modality combinations: Text & TTS in/Text out, TTS in/Text out and TTS in/Text & ASR out. The SoftBridge logged the conversations for subsequent analysis. The trial showed a largely successful conversation. Success factors include a) a text and computer literate DU who is familiar with research practise, b) using the system to explain the research as we conducted it and c) that the multi-modal bridging capabilities overcame the expected shortcomings of TTS and especially ASR. The lessons learned from this trial will be applied to the next trial once the necessary modifications have been implemented.Telkom, Siemens, THRI

SignSupport: a limited communication domain mobile aid for a Deaf patient at the pharmacy

This paper discusses a prototype for a communication aid on a mobile phone to support a Deaf1 person visiting a public hospital pharmacy. The aim is to prevent problems of non-compliance to treatment due to poor communication between a Deaf patient and a pharmacist. We studied the communication exchange between pharmacists and Deaf patients in a pharmacy setting in order to extract the most relevant content exchange between the two parties. A prototype was developed on a mobile phone and iteratively tested using role plays, questionnaires and focus groups with pharmacy students and Deaf participants. The prototype allows pharmacists to input text and make selections that provide detailed medical instructions in signed language to a Deaf patient. The prototype demonstrates the feasibility of encoding a limited communication flow on a mobile device, with carefully sequenced sign language videos that a Deaf patient can watch and understand in order to take medicine correctly.Telkom, Cisco, Aria Technologies, THRIP, SANPADDepartment of HE and Training approved lis

SignSupport: A Mobile Aid for Deaf People Learning Computer Literacy Skills

This paper discusses a prototype of a learning aid on a mobile phone to support Deaf people learning computer literacy skills. The aim is to allow Deaf people to learn at their own pace which in turn reduces the dependence on a teacher to allow weaker learners be assisted. We studied the classroom dynamics and teaching methods to extract how lesson content is delivered. This helped us develop an authoring tool to structure lesson content for the prototype. A prototype has been developed using South African Sign Language videos arranged according to the structure of pre-existing lessons. The technical goal was to implement the prototype on a mobile device and tie the resulting exported lesson content from the authoring tool to a series of signed language videos and images so that a Deaf person can teach him/herself computer literacy skills. Results from the user testing found the prototype successful in allowing Deaf users to learn at their own pace thereby reducing the dependence on the teacher