The Role of Emotional and Facial Expression in Synthesised Sign Language Avatars

This thesis explores the role that underlying emotional facial expressions might have in regards to understandability in sign language avatars. Focusing specifically on Irish Sign Language (ISL), we examine the Deaf community’s requirement for a visual-gestural language as well as some linguistic attributes of ISL which we consider fundamental to this research. Unlike spoken language, visual-gestural languages such as ISL have no standard written representation. Given this, we compare current methods of written representation for signed languages as we consider: which, if any, is the most suitable transcription method for the medical receptionist dialogue corpus. A growing body of work is emerging from the field of sign language avatar synthesis. These works are now at a point where they can benefit greatly from introducing methods currently used in the field of humanoid animation and, more specifically, the application of morphs to represent facial expression. The hypothesis underpinning this research is: augmenting an existing avatar (eSIGN) with various combinations of the 7 widely accepted universal emotions identified by Ekman (1999) to deliver underlying facial expressions, will make that avatar more human-like. This research accepts as true that this is a factor in improving usability and understandability for ISL users. Using human evaluation methods (Huenerfauth, et al., 2008) the research compares an augmented set of avatar utterances against a baseline set with regards to 2 key areas: comprehension and naturalness of facial configuration. We outline our approach to the evaluation including our choice of ISL participants, interview environment, and evaluation methodology. Remarkably, the results of this manual evaluation show that there was very little difference between the comprehension scores of the baseline avatars and those augmented withEFEs. However, after comparing the comprehension results for the synthetic human avatar “Anna” against the caricature type avatar “Luna”, the synthetic human avatar Anna was the clear winner. The qualitative feedback allowed us an insight into why comprehension scores were not higher in each avatar and we feel that this feedback will be invaluable to the research community in the future development of sign language avatars. Other questions asked in the evaluation focused on sign language avatar technology in a more general manner. Significantly, participant feedback in regard to these questions indicates a rise in the level of literacy amongst Deaf adults as a result of mobile technology

Utilization of Avatar-based Technology in The Area of Sign language... A Review

Information and communication technology (ICT) has progressed rapidly in recent years, and it is becoming necessary for everybody including deafpeople. This paper gives an overview of using a technology called Avatar-based technology in the area of sign language, which is the normal language of the deafworldwide, although it is different from country to another. This paper covers the basic concepts related to the signing avatar and the efforts for applying it indifferent sign language worldwide, especially Arabic Sign Language (ArSL)

Generating realistic, animated human gestures in order to model, analyse and recognize Irish Sign Language

The aim of this thesis is to generate a gesture recognition system which can recognize several signs of Irish Sign Language (ISL). This project is divided into three parts. The first part provides background information on ISL. An overview of the ISL structure is a prerequisite to identifying and understanding the difficulties encountered in the development of a recognition system. The second part involves the generation of a data repository: synthetic and real-time video. Initially the synthetic data is created in a 3D animation package in order to simplify the creation of motion variations of the animated signer. The animation environment in our implementation allows for the generation of different versions of the same gesture with slight variations in the parameters of the motion. Secondly a database of ISL real-time video was created. This database contains 1400 different signs, including motion variation in each gesture. The third part details step by step my novel classification system and the associated prototype recognition system. The classification system is constructed as a decision tree to identify each sign uniquely. The recognition system is based on only one component of the classification system and has been implemented as a Hidden Markov Model (HMM)

Jogos para o ensino de línguas de sinais/línguas orais escritas para surdos: uma revisão sistemática da literatura

This study carries out a survey about pieces of research which deal with games aiming at the teaching of sign languages and written oral languages to deaf people with the goal of identifying and characterizing the games which have been used for educational contexts. The researchers carried out a systematic review of literature using the Education Resources Information Center (ERIC), Web Science and Scopus database inserting the following terminologies in the wordsearch tool: ‘game’, ‘deaf’ and ‘sign language’. At first, a total of 530 studies were found, and based on a series of criteria, 31 papers were selected as a result of the search. The majority of the games dealt with in the papers are digital and one third of those utilize an electronic device for gesture recognition or sensor of some sort. Games in sign language or bilingual teaching represent most of the selected studies, even though it was observed that a good deal of those games teach only the manual alphabet or isolated sign from the respective sign languages. Furthermore, only a handful of those games make use of sign language in situations which simulate a real communicative context.Este estudio realiza una encuesta sobre investigaciones que abordan los juegos destinados a la enseñanza de lenguas de signos o lenguas orales escritos para sordos con el fin de mapear y caracterizar los juegos que se han utilizado en contextos educativos. Se realizó una revisión bibliográfica sistemática en las bases de datos Education Resources Information Center (ERIC), Web of Science y Scopus, utilizando como términos de búsqueda las palabras: game, deaf e sign language. En la primera búsqueda se encontraron un total de 530 artículos. La selección está basada en los criterios de inclusión y exclusión, que resultó en el análisis de 31 artículos. La mayoría de los juegos utilizados en las obras, son digitales y un tercio de ellos, utiliza algún dispositivo electrónico para reconocer gestos o sensores. Los juegos para la enseñanza de la lengua de signos o bilingües, representan la mayoría de los estudios seleccionados, sin embargo observamos que una buena parte enseña solo el alfabeto manual o signos aislados de las lenguas de signos. Pocos juegos han hecho uso de lenguas de signos en situaciones que simulan un contexto de comunicación real.Este estudo realiza um levantamento sobre pesquisas que abordam jogos voltados para o ensino de línguas de sinais ou de línguas orais escritas para surdos com objetivo de mapear e caracterizar os jogos que têm sido utilizados em contextos educacionais. Realizou-se uma revisão sistemática da literatura nas bases Education Resources Information Center (ERIC), Web of Science e Scopus, utilizando como termos de busca as palavras: game, deaf e sign language. Na primeira busca, foram encontrados um total de 530 artigos. A seleção com base nos critérios de inclusão e exclusão resultou na análise de 31 trabalhos. A maioria dos jogos utilizados nos trabalhos são digitais, e um terço deles faz uso de algum dispositivo eletrônico para reconhecimento de gestos ou sensores. Os jogos para ensino de língua de sinais ou bilíngues representam a maioria dos estudos selecionados, porém observamos que boa parte ensina apenas o alfabeto manual ou sinais isolados das línguas de sinais. Poucos jogos fizeram o uso das línguas de sinais em situações que simulem um contexto real de comunicação

New Method for Optimization of License Plate Recognition system with Use of Edge Detection and Connected Component

License Plate recognition plays an important role on the traffic monitoring and parking management systems. In this paper, a fast and real time method has been proposed which has an appropriate application to find tilt and poor quality plates. In the proposed method, at the beginning, the image is converted into binary mode using adaptive threshold. Then, by using some edge detection and morphology operations, plate number location has been specified. Finally, if the plat has tilt, its tilt is removed away. This method has been tested on another paper data set that has different images of the background, considering distance, and angel of view so that the correct extraction rate of plate reached at 98.66%.Comment: 3rd IEEE International Conference on Computer and Knowledge Engineering (ICCKE 2013), October 31 & November 1, 2013, Ferdowsi Universit Mashha

Guidelines for the design of a mobile phone application for deaf people

Deaf people in South Africa are no longer a minority group and their needs regarding communication and interaction must therefore be taken into consideration. This demographic does not communicate and interact in the same way as hearing people, which means that any existing usability or accessibility guidelines do not apply. As a result, Deaf people do not have access to information in the same way that a hearing person does. Providing proper access to Information and Communications Technology services, which meet their particular needs, will help the hearing impaired to better integrate into society. The importance of demolishing the communication barrier between the hearing and the hearing impaired is very important. Many people assume that sign language is somehow based on oral language. However, this is not the case. Sign languages are natural visual-spatial languages, and form a contrast with spoken languages which are auditory-vocal based. Acknowledging this fact in 2008, the South African Sign Language Policy Implementation Conference together with several governmental bodies suggested that South African Sign Language should become recognized as the 12th official language of South Africa. Due to the fact that cell phone technology continues to evolve, it will remain a tool of communication upon which Deaf people heavily rely on. The cell phone fulfils a basic need for everyone, but especially for the Deaf demographic as they rely on the short message system to communicate. Deaf people are currently faced with a lack of appropriate mobile phone applications, which would allow them to communicate with hearing as well as deaf people. The primary objective of this research was therefore to make mobile technology equally accessible meaning usable to members of the Deaf demographic. The focus of the research was to investigate the communication barrier and a range of variables that can influence the deaf user’s experience. Topics such as user interface design, usability and interaction were investigated. The outcome of this research was to propose a set of guidelines that, when applied to the design of a website or to phone application accessibility, would ensure communication and interaction from a deaf user. The proposed set of guidelines was then applied to the design of the high fidelity prototype of a mobile phone application. The specific application is a messaging phone application that allows deaf users to communicate with other deaf and hearing users via short message system. A mobile phone application that allows Deaf people to send and receive messages based on the sign language alphabet. The application was named Signchat. Purpose of this was to visibly display how the guidelines were implemented in Signchat. While Signchat’s main purpose is to accommodate the needs of Deaf people, it is also a learning tool and an application that bridges the gap by allowing deaf and hearing users to communicate

Guidelines for the design of a mobile phone application for deaf people

Deaf people in South Africa are no longer a minority group and their needs regarding communication and interaction must therefore be taken into consideration. This demographic does not communicate and interact in the same way as hearing people, which means that any existing usability or accessibility guidelines do not apply. As a result, Deaf people do not have access to information in the same way that a hearing person does. Providing proper access to Information and Communications Technology services, which meet their particular needs, will help the hearing impaired to better integrate into society. The importance of demolishing the communication barrier between the hearing and the hearing impaired is very important. Many people assume that sign language is somehow based on oral language. However, this is not the case. Sign languages are natural visual-spatial languages, and form a contrast with spoken languages which are auditory-vocal based. Acknowledging this fact in 2008, the South African Sign Language Policy Implementation Conference together with several governmental bodies suggested that South African Sign Language should become recognized as the 12th official language of South Africa. Due to the fact that cell phone technology continues to evolve, it will remain a tool of communication upon which Deaf people heavily rely on. The cell phone fulfils a basic need for everyone, but especially for the Deaf demographic as they rely on the short message system to communicate. Deaf people are currently faced with a lack of appropriate mobile phone applications, which would allow them to communicate with hearing as well as deaf people. The primary objective of this research was therefore to make mobile technology equally accessible meaning usable to members of the Deaf demographic. The focus of the research was to investigate the communication barrier and a range of variables that can influence the deaf user’s experience. Topics such as user interface design, usability and interaction were investigated. The outcome of this research was to propose a set of guidelines that, when applied to the design of a website or to phone application accessibility, would ensure communication and interaction from a deaf user. The proposed set of guidelines was then applied to the design of the high fidelity prototype of a mobile phone application. The specific application is a messaging phone application that allows deaf users to communicate with other deaf and hearing users via short message system. A mobile phone application that allows Deaf people to send and receive messages based on the sign language alphabet. The application was named Signchat. Purpose of this was to visibly display how the guidelines were implemented in Signchat. While Signchat’s main purpose is to accommodate the needs of Deaf people, it is also a learning tool and an application that bridges the gap by allowing deaf and hearing users to communicate

Assistive technologies for severe and profound hearing loss: beyond hearing aids and implants

Assistive technologies offer capabilities that were previously inaccessible to individuals with severe and profound hearing loss who have no or limited access to hearing aids and implants. This literature review aims to explore existing assistive technologies and identify what still needs to be done. It is found that there is a lack of focus on the overall objectives of assistive technologies. In addition, several other issues are identified i.e. only a very small number of assistive technologies developed within a research context have led to commercial devices, there is a predisposition to use the latest expensive technologies and a tendency to avoid designing products universally. Finally, the further development of plug-ins that translate the text content of a website to various sign languages is needed to make information on the internet more accessible

Development of Hearing-Impaired Evaluation Model for Mobile Phone

This paper discussed on usability evaluation model for the hearing-impaired mobile applications. Mobile applications developed for the disabled (hearing-impaired) are usually being scanted in term of usefulness and accessible for the disabled group. Evaluating the usability of such application are difficult due to the reason not many current models focus on the requirements for disabled people. Thus, this paper has proposed usability evaluation model for the hearing-impaired mobile application so that usability issue in such application could be identified and ensure applicability for daily usage. The model helps mobile developer and interface evaluator to produce better mobile applications for disabled people