274,559 research outputs found
Visual Recognition of Bengali Sign Language using Local Binary Pattern Compared with ANN
This paper presents an overview of visual recognition of Bengali Sign Language. In this paper we learn and detect a sequence of sign words and recognize the sign language that are understandable to the deaf and hearing impaired people to help normal people understand the meaning of these words. The research discusses the characteristics of the human sign languages, the requirements and difficulties behind visual sign recognition, how to deal with others persons and the different techniques used in the sign language recognition. The paper consists of two major parts, namely the learning part and the detection part. The system takes the sign images as its input. First sign images are learnt by the proposed system. When a sign image is given for recognition, the detection part identifies the image with the help of previously learned images. For learning and detection we have used local binary pattern compared with back propagation algorithm of Artificial Neural Network. We believe that this research will be of much help to express their thoughts and feelings between the deaf people and the normal people
Two-Stream Network for Sign Language Recognition and Translation
Sign languages are visual languages using manual articulations and non-manual
elements to convey information. For sign language recognition and translation,
the majority of existing approaches directly encode RGB videos into hidden
representations. RGB videos, however, are raw signals with substantial visual
redundancy, leading the encoder to overlook the key information for sign
language understanding. To mitigate this problem and better incorporate domain
knowledge, such as handshape and body movement, we introduce a dual visual
encoder containing two separate streams to model both the raw videos and the
keypoint sequences generated by an off-the-shelf keypoint estimator. To make
the two streams interact with each other, we explore a variety of techniques,
including bidirectional lateral connection, sign pyramid network with auxiliary
supervision, and frame-level self-distillation. The resulting model is called
TwoStream-SLR, which is competent for sign language recognition (SLR).
TwoStream-SLR is extended to a sign language translation (SLT) model,
TwoStream-SLT, by simply attaching an extra translation network.
Experimentally, our TwoStream-SLR and TwoStream-SLT achieve state-of-the-art
performance on SLR and SLT tasks across a series of datasets including
Phoenix-2014, Phoenix-2014T, and CSL-Daily.Comment: Accepted by NeurIPS 202
Automatic recognition of fingerspelled words in British Sign Language
We investigate the problem of recognizing words from
video, fingerspelled using the British Sign Language (BSL)
fingerspelling alphabet. This is a challenging task since the
BSL alphabet involves both hands occluding each other, and
contains signs which are ambiguous from the observerâs
viewpoint. The main contributions of our work include:
(i) recognition based on hand shape alone, not requiring
motion cues; (ii) robust visual features for hand shape
recognition; (iii) scalability to large lexicon recognition
with no re-training.
We report results on a dataset of 1,000 low quality webcam
videos of 100 words. The proposed method achieves a
word recognition accuracy of 98.9%
Dictionary-based lip reading classification
Visual lip reading recognition is an essential stage in many multimedia systems such as âAudio Visual Speech
Recognitionâ [6], âMobile Phone Visual System for deaf peopleâ, âSign Language Recognition Systemâ, etc.
The use of lip visual features to help audio or hand recognition is appropriate because this information is robust
to acoustic noise. In this paper, we describe our work towards developing a robust technique for lip reading
classification that extracts the lips in a colour image by using EMPCA feature extraction and k-nearest-neighbor
classification. In order to reduce the dimensionality of the feature space the lip motion is characterized by three
templates that are modelled based on different mouth shapes: closed template, semi-closed template, and wideopen
template. Our goal is to classify each image sequence based on the distribution of the three templates and
group the words into different clusters. The words that form the database were grouped into three different
clusters as follows: group1(âIâ, âhighâ, âlieâ, âhardâ, âcardâ, âbyeâ), group2(âyou, âoweâ, âwordâ), group3(âbirdâ)
A survey on mouth modeling and analysis for Sign Language recognition
© 2015 IEEE.Around 70 million Deaf worldwide use Sign Languages (SLs) as their native languages. At the same time, they have limited reading/writing skills in the spoken language. This puts them at a severe disadvantage in many contexts, including education, work, usage of computers and the Internet. Automatic Sign Language Recognition (ASLR) can support the Deaf in many ways, e.g. by enabling the development of systems for Human-Computer Interaction in SL and translation between sign and spoken language. Research in ASLR usually revolves around automatic understanding of manual signs. Recently, ASLR research community has started to appreciate the importance of non-manuals, since they are related to the lexical meaning of a sign, the syntax and the prosody. Nonmanuals include body and head pose, movement of the eyebrows and the eyes, as well as blinks and squints. Arguably, the mouth is one of the most involved parts of the face in non-manuals. Mouth actions related to ASLR can be either mouthings, i.e. visual syllables with the mouth while signing, or non-verbal mouth gestures. Both are very important in ASLR. In this paper, we present the first survey on mouth non-manuals in ASLR. We start by showing why mouth motion is important in SL and the relevant techniques that exist within ASLR. Since limited research has been conducted regarding automatic analysis of mouth motion in the context of ALSR, we proceed by surveying relevant techniques from the areas of automatic mouth expression and visual speech recognition which can be applied to the task. Finally, we conclude by presenting the challenges and potentials of automatic analysis of mouth motion in the context of ASLR
Improving Continuous Sign Language Recognition with Cross-Lingual Signs
This work dedicates to continuous sign language recognition (CSLR), which is
a weakly supervised task dealing with the recognition of continuous signs from
videos, without any prior knowledge about the temporal boundaries between
consecutive signs. Data scarcity heavily impedes the progress of CSLR. Existing
approaches typically train CSLR models on a monolingual corpus, which is orders
of magnitude smaller than that of speech recognition. In this work, we explore
the feasibility of utilizing multilingual sign language corpora to facilitate
monolingual CSLR. Our work is built upon the observation of cross-lingual
signs, which originate from different sign languages but have similar visual
signals (e.g., hand shape and motion). The underlying idea of our approach is
to identify the cross-lingual signs in one sign language and properly leverage
them as auxiliary training data to improve the recognition capability of
another. To achieve the goal, we first build two sign language dictionaries
containing isolated signs that appear in two datasets. Then we identify the
sign-to-sign mappings between two sign languages via a well-optimized isolated
sign language recognition model. At last, we train a CSLR model on the
combination of the target data with original labels and the auxiliary data with
mapped labels. Experimentally, our approach achieves state-of-the-art
performance on two widely-used CSLR datasets: Phoenix-2014 and Phoenix-2014T.Comment: Accepted by ICCV 202
Visual recognition of American sign language using hidden Markov models
Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1995.Includes bibliographical references (leaves 48-52).by Thad Eugene Starner.M.S
Visual Recognition of Bengali Sign Language using Local Binary Pattern Compared with ANN
This paper presents an overview of visual recognition of Bengali Sign Language. In this paper we learn and detect a sequence of sign words and recognize the sign language that are understandable to the deaf and hearing impaired people to help normal people understand the meaning of these words. The research discusses the characteristics of the human sign languages, the requirements and difficulties behind visual sign recognition, how to deal with others persons and the different techniques used in the sign language recognition. The paper consists of two major parts, namely the learning part and the detection part. The system takes the sign images as its input. First sign images are learnt by the proposed system. When a sign image is given for recognition, the detection part identifies the image with the help of previously learned images. For learning and detection we have used local binary pattern compared with back propagation algorithm of Artificial Neural Network. We believe that this research will be of much help to express their thoughts and feelings between the deaf people and the normal people
- âŠ