The use of mobile phones as service-delivery devices in sign language machine translation system

Masters of ScienceThis thesis investigates the use of mobile phones as service-delivery devices in a sign language machine translation system. Four sign language visualization methods were evaluated on mobile phones. Three of the methods were synthetic sign language visualization methods. Three factors were considered: the intelligibility of sign language, as rendered by the method; the power consumption; and the bandwidth usage associated with each method. The average intelligibility rate was 65%, with some methods achieving intelligibility rates of up to 92%. The average size was 162 KB and, on average, the power consumption increased to 180% of the idle state, across all methods. This research forms part of the Integration of Signed and Verbal Communication: South African Sign Language Recognition and Animation (SASL) project at the University of the Western Cape and serves as an integration platform for the group's research. In order to perform this research a machine translation system that uses mobile phones as service-delivery devices was developed as well as a 3D Avatar for mobile phones. It was concluded that mobile phones are suitable service-delivery platforms for sign language machine translation systems.South Afric

The design of a generic signing avatar animation system

Thesis (MScIng)--University of Stellenbosch, 2006.ENGLISH ABSTRACT: We designed a generic avatar animator for use in sign language related projects. The animator is capable of animating any given avatar that is compliant with the H-Anim standard for humanoid animation. The system was designed with the South African Sign Language Machine Translation (SASL-MT) project in mind, but can easily be adapted to other sign language projects due to its generic design. An avatar that is capable of accurately performing sign language gestures is a special kind of avatar and is referred to as a signing avatar. In this thesis we investigate the special characteristics of signing avatars and address the issue of finding a generic design for the animation of such an avatar.AFRIKAANSE OPSOMMING: Ons het ’n generiese karakteranimasiestelsel ontwikkel vir gebruik in gebaretaal verwante projekte. Die animasiestelsel het die vermo¨e om enige karaktermodel wat met die H-Anim standaard versoenbaar is, te animeer. Die animasiestelsel is ontwerp met die oog op gebruik in die South African Sign Language Machine Translation (SASL-MT) projek, maar kan maklik aangepas word vir ander gebaretaalprojekte te danke aan die generiese ontwerp. ’n Karaktermodel wat in staat is om gebare akkuraat te maak is ’n spesiale tipe karaktermodel wat bekend staan as ’n gebaretaal avatar (Engels : signing avatar). In hierdie tesis ondersoek ons die spesiale eienskappe van ’n gebaretaal avatar en beskou die soektog na ’n generiese ontwerp vir die animering van so ’n karaktermodel

Avatar animation from SignWriting notation

>Magister Scientiae - MScThe SASL project at the University of the Western Cape is in the process of developing a machine translation system that can translate fully-fledged phrases between South African Sign Language (SASL) and English in real-time.To visualise sign language,the system aims to make use of a 3D humanoid avatar created by van Wyk. Moemedi used this avatar to create an animation system that visualises a small set of simple Phrases from very simple SignWriting notation input. This research aims to achieve an animation system that can render full sign language sentences given complex SignWriting notation glyphs with multiple sections. The specific focus of the research is achieving animations that are accurate representations of the SignWriting input in terms of the five fundamental parameters of sign language, namely, hand motion, location, orientation and shape, as well as non-manual features such as facial expressions. An experiment Was carried out to determine the accuracy of the proposed system on a set of 20 SASL phrases annotated with SignWriting notation. It was found that the proposed system is highly accurate, achieving an average accuracy of 81.6%

An example-based approach to translating sign language

Users of sign languages are often forced to use a language in which they have reduced competence simply because documentation in their preferred format is not available. While some research exists on translating between natural and sign languages, we present here what we believe to be the first attempt to tackle this problem using an example-based (EBMT) approach. Having obtained a set of English–Dutch Sign Language examples, we employ an approach to EBMT using the ‘Marker Hypothesis’ (Green, 1979), analogous to the successful system of (Way & Gough, 2003), (Gough & Way, 2004a) and (Gough & Way, 2004b). In a set of experiments, we show that encouragingly good translation quality may be obtained using such an approach

Lost in translation: the problems of using mainstream MT evaluation metrics for sign language translation

In this paper we consider the problems of applying corpus-based techniques to minority languages that are neither politically recognised nor have a formally accepted writing system, namely sign languages. We discuss the adoption of an annotated form of sign language data as a suitable corpus for the development of a data-driven machine translation (MT) system, and deal with issues that arise from its use. Useful software tools that facilitate easy annotation of video data are also discussed. Furthermore, we address the problems of using traditional MT evaluation metrics for sign language translation. Based on the candidate translations produced from our example-based machine translation system, we discuss why standard metrics fall short of providing an accurate evaluation and suggest more suitable evaluation methods

Combining data-driven MT systems for improved sign language translation

In this paper, we investigate the feasibility of combining two data-driven machine translation (MT) systems for the translation of sign languages (SLs). We take the MT systems of two prominent data-driven research groups, the MaTrEx system developed at DCU and the Statistical Machine Translation (SMT) system developed at RWTH Aachen University, and apply their respective approaches to the task of translating Irish Sign Language and German Sign Language into English and German. In a set of experiments supported by automatic evaluation results, we show that there is a definite value to the prospective merging of MaTrEx’s Example-Based MT chunks and distortion limit increase with RWTH’s constraint reordering

Computerization of African languages-French dictionaries

This paper relates work done during the DiLAF project. It consists in converting 5 bilingual African language-French dictionaries originally in Word format into XML following the LMF model. The languages processed are Bambara, Hausa, Kanuri, Tamajaq and Songhai-zarma, still considered as under-resourced languages concerning Natural Language Processing tools. Once converted, the dictionaries are available online on the Jibiki platform for lookup and modification. The DiLAF project is first presented. A description of each dictionary follows. Then, the conversion methodology from .doc format to XML files is presented. A specific point on the usage of Unicode follows. Then, each step of the conversion into XML and LMF is detailed. The last part presents the Jibiki lexical resources management platform used for the project.Comment: 8 page

AfriSign: Machine Translation for African Sign Languages

Sign language translation is an active area of research with the main goal of bridging the communication gap between deaf and hearing individuals. In Natural Language Processing (NLP), there is a growing interest in this task, leading to new datasets and research on translation approaches. But while there has been significant progress for sign languages from high-income countries, minimal research has been conducted on African sign language translation. In this paper, we curate a novel dataset of African sign languages, with a focus on machine translation as the main application. The dataset contains English Bible verses and videos with translations into six different African sign languages. Using this dataset, we report experiments on African sign language machine translation, including baseline Transformer systems, multilingual training and cross-lingual transfer learning