Spanish Sign Language synthesis system

This is the author’s version of a work that was accepted for publication in Journal of Visual Languages and Computing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of Visual Languages and Computing,23, 3, (2012) DOI: 10.1016/j.jvlc.2012.01.003This work presents a new approach to the synthesis of Spanish Sign Language (LSE). Its main contributions are the use of a centralized relational database for storing sign descriptions, the proposal of a new input notation and a new avatar design, the skeleton structure of which improves the synthesis process. The relational database facilitates a highly detailed phonologic description of the signs that include parameter synchronization and timing. The centralized database approach has been introduced to allow the representation of each sign to be validated by the LSE National Institution, FCNSE. The input notation, designated HLSML, presents multiple levels of abstraction compared with current input notations. Redesigned input notation is used to simplify the description and the manual definition of LSE messages. Synthetic messages obtained using our approach have been evaluated by deaf users; in this evaluation a maximum recognition rate of 98.5% was obtained for isolated signs and a recognition rate of 95% was achieved for signed sentences

An on-line system adding subtitles and sign language to Spanish audio-visual content

Deaf people cannot properly access the speech information stored in any kind of recording format (audio, video, etc). We present a system that provides with subtitling and Spanish Sign Language representation capabilities to allow Spanish Deaf population can access to such speech content. The system is composed by a speech recognition module, a machine translation module from Spanish to Spanish Sign Language and a Spanish Sign Language synthesis module. On the deaf person side, a user-friendly interface with subtitle and avatar components allows him/her to access the speech information

Distributed Spanish Sign Language synthesizer architectures

This is an electronic version of the paper presented at the Congreso Internacional de Interacción Persona-Ordenador, held in Bercelona on 2009This work presents the design of a distributed Sign Language synthesis architecture. The main objective of this design is to adapt the synthesis process to the diversity of the user devices. The synthesis process has been divided into several independent modules that can be executed either in a synthesis server or in the client device. Depending on the modules assigned to the server or the client, four different scenarios have been defined. These scenarios may vary from a heavy client design which executes the whole synthesis, to a light client design similar to a video player. These four scenarios will provide the maximum signed message quality independently of the device hardware resource

A Spanish Sign Language (LSE) Adaptation of the Communicative Development Inventories

This article presents the adaptation of the MacArthur Communicative Development Inventory (CDI; Fenson et al., 1993, 1994) to Spanish Sign Language (LSE). Data were collected from 55 participants (32 boys and 23 girls; 17 deaf signers, 38 hearing signers) who, evaluated by their caregivers every four months, presented a total of 170 records. The parents reported the signs that the children could understand or produce between 8-36 months. Results suggested that the CDI adapted to LSE is a valid and reliable instrument. Signing children could understand more signs than they produced at this early developmental stage. There were no significant differences between boys and girls, or between deaf and hearing children. The development of LSE is similar to other sign languages, although with a lower production of signs in the early stages, perhaps due to the bilingualism of most of the children of our study.Se presenta la adaptación a la lengua de signos española (LSE) del Inventario de Desarrollo Comunicativo MacArthur (CDI; Fenson et al., 1993, 1994). Se recogieron datos de 55 participantes (32 niños y 23 niñas; 17 niños y niñas sordos signantes y 38 oyentes signantes), que, evaluados cada cuatro meses por sus cuidadores, representaron un total de 170 registros. Los padres y madres informaron de los signos que los niños podían producir o comprender entre los 8 y 36 meses. Los resultados sugieren que el CDI adaptado a la LSE es un instrumento válido y fiable. Los niños signantes podían entender más signos que producir en esta etapa del desarrollo temprano. No hubo diferencias significativas entre niños y niñas o entre niños sordos y oyentes. El desarrollo de la LSE es similar a otras lenguas de signos, aunque con una menor producción de signos en las etapas iniciales, quizás debido al bilingüismo de la mayoría de los niños del estudio.Consejería de Innovación, Ciencia y Empresa, Junta de Andalucía (España), Proyectos de Excelencia (P11-SEJ-7417)

Hybrid paradigm for Spanish Sign Language synthesis

The final publication is available at Springer via http://dx.doi.org/10.1007/s10209-011-0245-9This work presents a hybrid approach to sign language synthesis. This approach allows the hand-tuning of the phonetic description of the signs, which focuses on the time aspect of the sign. Therefore, the approach retains the capacity for the performing of morpho-phonological operations, like notation-based approaches, and improves the synthetic signing performance, such as the hand-tuned animations approach. The proposed approach simplifies the input message description using a new high-level notation and storage of sign phonetic descriptions in a relational database. Such relational database allows for more flexible sign phonetic descriptions; it also allows for a description of sign timing and the synchronization between sign phonemes. The new notation, named HLSML, is a gloss-based notation focusing on message description in it. HLSML introduces several tags that allow for the modification of the signs in the message that defines dialect and mood variations, both of which are defined in the relational database, and message timing, including transition durations and pauses. A new avatar design is also proposed that simplifies the development of the synthesizer and avoids any interference with the independence of the sign language phonemes during animation. The obtained results showed an increase of the sign recognition rate compared to other approaches. This improvement was based on the active role that the sign language experts had in the description of signs, which was the result of the flexibility of the sign storage approach. The approach will simplify the description of synthesizable signed messages, thus facilitating the creation of multimedia-signed contents

Spanish generation from Spanish Sign Language using a phrase-based translation system

This paper describes the development of a Spoken Spanish generator from Spanish Sign Language (LSE – Lengua de Signos Española) in a specific domain: the renewal of Identity Document and Driver’s license. The system is composed of three modules. The first one is an interface where a deaf person can specify a sign sequence in sign-writing. The second one is a language translator for converting the sign sequence into a word sequence. Finally, the last module is a text to speech converter. Also, the paper describes the generation of a parallel corpus for the system development composed of more than 4,000 Spanish sentences and their LSE translations in the application domain. The paper is focused on the translation module that uses a statistical strategy with a phrase-based translation model, and this paper analyses the effect of the alignment configuration used during the process of word based translation model generation. Finally, the best configuration gives a 3.90% mWER and a 0.9645 BLEU

Integration of a talking head into a Spanish Sign Language synthesizer

This is an electronic version of the paper presented at the Congreso Internacional de Interacción Persona-Ordenador, held in Bercelona on 2009In this paper, we present an integration of a talking head within a Spanish Sign Language synthesizer. The whole system consists of three different steps: First, the input acoustic signal is transformed into a sequence of phones by means of a speech recognition process. This sequence of phones is mapped in a second step to a sequence of visemes and finally, the resulting sequence of visemes is played by means of a talking head integrated into the avatar used in the Spanish Sign Language synthesizer

Language Resources for Spanish - Spanish Sign Language (LSE) translation

This paper describes the development of a Spanish Spanish Sign Language (LSE) translation system. Firstly, it describes the first Spanish Spanish Sign Language (LSE) parallel corpus focused on two specific domains: the renewal of the Identity Document and Driver’s License. This corpus includes more than 4,000 Spanish sentences (in these domains), their LSE translation and a video for each LSE sentence with the sign language representation. This corpus also contains more than 700 sign descriptions in several sign writing specifications. The translation system developed with this corpus consists of two modules: a Spanish into LSE translation module that is composed of a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of signs) and a 3D avatar animation module (for playing back the signs). The second module is a Spanish generator from LSE made up of a visual interface (for specifying a sequence of signs in sign writing), a language translator (for generating the sequence of words in Spanish) and a text to speech converter. For each language translation, the system uses three technologies: an example based strategy, a rule based translation method and a statistical translator

Automatic Categorization for Improving Spanish into Spanish Sign Language Machine Translation

This paper describes a preprocessing module for improving the performance of a Spanish into Spanish Sign Language (Lengua de Signos Espanola: LSE) translation system when dealing with sparse training data. This preprocessing module replaces Spanish words with associated tags. The list with Spanish words (vocabulary) and associated tags used by this module is computed automatically considering those signs that show the highest probability of being the translation of every Spanish word. This automatic tag extraction has been compared to a manual strategy achieving almost the same improvement. In this analysis, several alternatives for dealing with non-relevant words have been studied. Non-relevant words are Spanish words not assigned to any sign. The preprocessing module has been incorporated into two well-known statistical translation architectures: a phrase-based system and a Statistical Finite State Transducer (SFST). This system has been developed for a specific application domain: the renewal of Identity Documents and Driver's License. In order to evaluate the system a parallel corpus made up of 4080 Spanish sentences and their LSE translation has been used. The evaluation results revealed a significant performance improvement when including this preprocessing module. In the phrase-based system, the proposed module has given rise to an increase in BLEU (Bilingual Evaluation Understudy) from 73.8% to 81.0% and an increase in the human evaluation score from 0.64 to 0.83. In the case of SFST, BLEU increased from 70.6% to 78.4% and the human evaluation score from 0.65 to 0.82

Spanish sign language interpreter for mexican linguistics

We present here the first visual interface for a Mexican Spanish Sign Language translator on its first development stage: sign-writing recognition. The software was developed for the unique characteristics of Mexican linguistics and was designed in order to use sentences or a sequence of signs in sign-writing system which are decoded by the program and converted into a series of images with movement that correspond to the Mexican sign language system. Using a lexical, syntactic and semantic algorithms plus free software such as APIs's from Java, video converter software, data base manager like MySQL, Postgres and SQlite, was possible to read and interpret the rich and complex Mexican language. Our application for visual interface showed to be capable of reading and reconstruct each sentence used for the interpreter and translate it into a high de finition video. The average time of video display vs number of sentences to interpret, probed to be in linear relation with an average time of two seconds per sentence. The software has over come the problem of homonym words frequently used in Spanish language and verb tense relation for each sentence, special symbols such as #, %, $, etc. are still not recognized into the softwareFacultad de Informátic