Automatic recognition of fingerspelled words in British Sign Language

We investigate the problem of recognizing words from video, fingerspelled using the British Sign Language (BSL) fingerspelling alphabet. This is a challenging task since the BSL alphabet involves both hands occluding each other, and contains signs which are ambiguous from the observer’s viewpoint. The main contributions of our work include: (i) recognition based on hand shape alone, not requiring motion cues; (ii) robust visual features for hand shape recognition; (iii) scalability to large lexicon recognition with no re-training. We report results on a dataset of 1,000 low quality webcam videos of 100 words. The proposed method achieves a word recognition accuracy of 98.9%

A kinematic analysis of hand configurations in static and dynamic fingerspelling

The focus of this study was the investigation of target handshapes in American Sign Language fingerspelling in order to determine whether there was a difference between static canonical structures and structures produced in the context of a movement sequence. This was achieved by measuring the joint angles of a signing hand with an 18-sensor CyberGlove® by Virtual Technologies, Inc. A discriminant analysis was used to identify targets that occurred at points of minimum angular joint velocity. A multivariate analysis of variance with planned compansons was then applied to these dynamic data along with the static data to test the hypothesis. The results showed that there was a significant difference between handshapes produced statically and those produced dynamically, which suggested that a simple, cipher model of static handshapes produced within the context of a movement sequence is not sufficient to account for the production and perception of fingerspelling. These findings may be applied to future research in sign language recognition, so that consideration of the variability of target handshapes, as influenced by the spatiotemporal environment, might be incorporated into future models

Master of Arts

thesisFingerspelling in American Sign Language (ASL) is a system in which 26 onehanded signs represent the letters of the English alphabet and are formed sequentially to spell out words borrowed from oral languages or letter sequences. Patrie and Johnson have proposed a distinction in fingerspelling styles between careful fingerspelling and rapid fingerspelling, which appear to correspond to clear speech and plain speech styles. The criteria for careful fingerspelling include indexing of fingerspelled words, completely spelled words, limited coarticulation, a slow signing rate, and even rhythm, while rapid fingerspelling involves lack of indexing, increased dropping of letters, coarticulation, a faster signing rate, and the first and last letter of the words being held longer. They further propose that careful fingerspelling is used for initial uses of all fingerspelled words in running signing, with rapid fingerspelling being used for second and further mentions of fingerspelled words. I examine the 45 fingerspelled content words in a speech given by a Deaf native signer using quantitative measures, including a Coarticulation Index that permits comparing the degree of coarticulation in different words. I find that first mentions are more hyperarticulated than second mentions but that not all first mentions are hyperarticulated to the same extent and that topicality of the words may have bearing on this. I also show that the reduction of fingerspelled words is consistent with the reduction seen in repeated words in spoken English

BdSpell: A YOLO-based Real-time Finger Spelling System for Bangla Sign Language

In the domain of Bangla Sign Language (BdSL) interpretation, prior approaches often imposed a burden on users, requiring them to spell words without hidden characters, which were subsequently corrected using Bangla grammar rules due to the missing classes in BdSL36 dataset. However, this method posed a challenge in accurately guessing the incorrect spelling of words. To address this limitation, we propose a novel real-time finger spelling system based on the YOLOv5 architecture. Our system employs specified rules and numerical classes as triggers to efficiently generate hidden and compound characters, eliminating the necessity for additional classes and significantly enhancing user convenience. Notably, our approach achieves character spelling in an impressive 1.32 seconds with a remarkable accuracy rate of 98\%. Furthermore, our YOLOv5 model, trained on 9147 images, demonstrates an exceptional mean Average Precision (mAP) of 96.4\%. These advancements represent a substantial progression in augmenting BdSL interpretation, promising increased inclusivity and accessibility for the linguistic minority. This innovative framework, characterized by compatibility with existing YOLO versions, stands as a transformative milestone in enhancing communication modalities and linguistic equity within the Bangla Sign Language community

Thai Finger-Spelling Recognition Using a Cascaded Classifier Based on Histogram of Orientation Gradient Features

Hand posture recognition is an essential module in applications such as human-computer interaction (HCI), games, and sign language systems, in which performance and robustness are the primary requirements. In this paper, we proposed automatic classification to recognize 21 hand postures that represent letters in Thai finger-spelling based on Histogram of Orientation Gradient (HOG) feature (which is applied with more focus on the information within certain region of the image rather than each single pixel) and Adaptive Boost (i.e., AdaBoost) learning technique to select the best weak classifier and to construct a strong classifier that consists of several weak classifiers to be cascaded in detection architecture. We collected 21 static hand posture images from 10 subjects for testing and training in Thai letters finger-spelling. The parameters for the training process have been adjusted in three experiments, false positive rates (FPR), true positive rates (TPR), and number of training stages (N), to achieve the most suitable training model for each hand posture. All cascaded classifiers are loaded into the system simultaneously to classify different hand postures. A correlation coefficient is computed to distinguish the hand postures that are similar. The system achieves approximately 78% accuracy on average on all classifier experiments

Spanish Sign Language synthesis system

This is the author’s version of a work that was accepted for publication in Journal of Visual Languages and Computing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of Visual Languages and Computing,23, 3, (2012) DOI: 10.1016/j.jvlc.2012.01.003This work presents a new approach to the synthesis of Spanish Sign Language (LSE). Its main contributions are the use of a centralized relational database for storing sign descriptions, the proposal of a new input notation and a new avatar design, the skeleton structure of which improves the synthesis process. The relational database facilitates a highly detailed phonologic description of the signs that include parameter synchronization and timing. The centralized database approach has been introduced to allow the representation of each sign to be validated by the LSE National Institution, FCNSE. The input notation, designated HLSML, presents multiple levels of abstraction compared with current input notations. Redesigned input notation is used to simplify the description and the manual definition of LSE messages. Synthetic messages obtained using our approach have been evaluated by deaf users; in this evaluation a maximum recognition rate of 98.5% was obtained for isolated signs and a recognition rate of 95% was achieved for signed sentences