Automatic recognition of Arabic alphabets sign language using deep learning

Technological advancements are helping people with special needs overcome many communications’ obstacles. Deep learning and computer vision models are innovative leaps nowadays in facilitating unprecedented tasks in human interactions. The Arabic language is always a rich research area. In this paper, different deep learning models were applied to test the accuracy and efficiency obtained in automatic Arabic sign language recognition. In this paper, we provide a novel framework for the automatic detection of Arabic sign language, based on transfer learning applied on popular deep learning models for image processing. Specifically, by training AlexNet, VGGNet and GoogleNet/Inception models, along with testing the efficiency of shallow learning approaches based on support vector machine (SVM) and nearest neighbors algorithms as baselines. As a result, we propose a novel approach for the automatic recognition of Arabic alphabets in sign language based on VGGNet architecture which outperformed the other trained models. The proposed model is set to present promising results in recognizing Arabic sign language with an accuracy score of 97%. The suggested models are tested against a recent fully-labeled dataset of Arabic sign language images. The dataset contains 54,049 images, which is considered the first large and comprehensive real dataset of Arabic sign language to the furthest we know

Recognition of Bangladeshi Sign Language (BdSL) Words using Deep Convolutional Neural Networks (DCNNs)

In a world where effective communication is fundamental, individuals who are Deaf and Dumb (D&D) often face unique challenges due to their primary mode of communication—sign language. Despite the interpreters' invaluable roles, their lack of availability causes communication difficulties for the D&D individuals. This study explores whether the field of Human-Computer Interaction (HCI) could be a potential solution. The primary objective is to assist D&D individuals with computer applications that could act as mediators to bridge the communication gap between them and the wider hearing population. To ensure their independent communication, we propose an automated system that could detect specific Bangla Sign Language (BdSL) words, addressing a critical gap in the sign language detection and recognition literature. Our approach leverages deep learning and transfer learning principles to convert webcam-captured hand gestures into textual representations in real-time. The model's development and assessment rest upon 992 images created by the authors, categorized into ten distinct classes representing various BdSL words. Our findings show the DenseNet201 and ResNet50-V2 models achieve promising training and testing accuracies of 99% and 93%, respectively. Doi: 10.28991/ESJ-2023-07-06-019 Full Text: PD

Hand Gestures Recognition for Human-Machine Interfaces: A Low-Power Bio-Inspired Armband

Hand gesture recognition has recently increased its popularity as Human-Machine Interface (HMI) in the biomedical field. Indeed, it can be performed involving many different non-invasive techniques, e.g., surface ElectroMyoGraphy (sEMG) or PhotoPlethysmoGraphy (PPG). In the last few years, the interest demonstrated by both academia and industry brought to a continuous spawning of commercial and custom wearable devices, which tried to address different challenges in many application fields, from tele-rehabilitation to sign language recognition. In this work, we propose a novel 7-channel sEMG armband, which can be employed as HMI for both serious gaming control and rehabilitation support. In particular, we designed the prototype focusing on the capability of our device to compute the Average Threshold Crossing (ATC) parameter, which is evaluated by counting how many times the sEMG signal crosses a threshold during a fixed time duration (i.e., 130 ms), directly on the wearable device. Exploiting the event-driven characteristic of the ATC, our armband is able to accomplish the on-board prediction of common hand gestures requiring less power w.r.t. state of the art devices. At the end of an acquisition campaign that involved the participation of 26 people, we obtained an average classifier accuracy of 91.9% when aiming to recognize in real time 8 active hand gestures plus the idle state. Furthermore, with 2.92mA of current absorption during active functioning and 1.34mA prediction latency, this prototype confirmed our expectations and can be an appealing solution for long-term (up to 60 h) medical and consumer applications

A Comprehensive Review of Data-Driven Co-Speech Gesture Generation

Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co-speech gestures is a long-standing problem in computer animation and is considered an enabling technology in film, games, virtual social spaces, and for interaction with social robots. The problem is made challenging by the idiosyncratic and non-periodic nature of human co-speech gesture motion, and by the great diversity of communicative functions that gestures encompass. Gesture generation has seen surging interest recently, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep-learning-based generative models, that benefit from the growing availability of data. This review article summarizes co-speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule-based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text, and non-linguistic input. We also chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method. Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human-like motion; grounding the gesture in the co-occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.Comment: Accepted for EUROGRAPHICS 202

Recurrent Neural Networks with Weighting Loss for Early Prediction of Hand Movements

We propose in this work an approach for early prediction of hand movements using recurrent neural networks (RNNs) and a novel weighting loss. The proposed loss function leverages the outputs of an RNN at different time steps and weights their contributions to the final loss linearly over time steps. Additionally, a sample weighting scheme also constitutes a part of the weighting loss to deal with the scarcity of the samples where a change of hand movements takes place. The experiments conducted with the Ninapro database reveal that our proposed approach not only improves the performance in the early prediction task but also obtains state of the art results in classification of hand movements. These results are especially promising for the amputees

The Disabled Child

When children are born with disabilities or become disabled in childhood, parents often experience bewilderment: they find themselves unexpectedly in another world, without a roadmap, without community, and without narratives to make sense of their experiences. The Disabled Child: Memoirs of a Normal Future tracks the narratives that have emerged from the community of parent-memoirists who, since the 1980s, have written in resistance of their children’s exclusion from culture. Though the disabilities represented in the genre are diverse, the memoirs share a number of remarkable similarities; they are generally written by white, heterosexual, middle or upper-middle class, ablebodied parents, and they depict narratives in which the disabled child overcomes barriers to a normal childhood and adulthood. Apgar demonstrates that in the process of telling these stories, which recuperate their children as productive members of society, parental memoirists write their children into dominant cultural narratives about gender, race, and class. By reinforcing and buying into these norms, Apgar argues, “special needs” parental memoirs reinforce ableism at the same time that they’re writing against it

Beyond integration: reformulating physical disability in dance

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Doctor of Philosophy JulyDance performance that is inclusive of dancers with differing corporealities has the potential to generate positive societal change with regard to perceptions of physical difference. Dance is a valuable site for exploring the placement of the physically disabled body in contemporary society, and for disrupting existing perceptions of disability as transgressive. This can come about through the embodied presence of both dancer and viewer, entering into a relationship grounded in intersubjectivity, without having to rely on symbolic signification. This thesis examines the placement of disabled bodies in dance performance from the intersecting perspectives of Critical Disability Studies, Performance Studies and Interpersonal Neurobiology in order to formulate a framework for theorizing perceptions of disability, the act of viewing dance and the impact of choreographic intent on viewers’ perceptions of physical difference. In the first section, the sociopolitical placing of disabled bodies in western society is interrogated and a historiological study of both disability identity and the emergence of integrated dance is critically analysed. The second section provides detailed analyses of three dance performances that are inclusive of dancers with physical disabilities: GIMP (2009), Heidi Latsky, Diagnosis of a Faun (2009) Tamar Rogoff, and water burns sun (2009) Petra Kuppers. Each represents a specific understanding of disability, creating an evolutionary framework for conceptualizing different perceptions of disabled bodies as either monstrous freak, heroic victim or corporeally diverse. The third section creates connections between new knowledge in interpersonal neurobiology and viewers' perceptions of disability that are activated through viewing dance performance, thus providing an understanding of the mechanisms of discrimination and marginalization of people who embody difference, as well as uncovering mechanisms that have the potential to be reparative. The application of neuroscientific knowledge to Performance Studies can be modulated and expanded by considering the interpersonal communicative dimension of dance performance that is inclusive of differing corporealities. A theoretical approach that encompasses the neuroscientific conceptualization of intersubjectivity in creating empathic attunement between viewer and dancer, can offer a means of understanding the innate potential of dance performance to bring about societal change

Hand Gesture Based Surveillance Robot

In this work, a hardware and software based integrated system is developed for hand gesture based surveillance robot. The proposed system is a non-invasive technique and software part of the system uses gesture based image processing technique. The hardware part is developed based on AVR microcontroller platform. The captured image of hand is segmented and its contour is determined. The convexity defects are computed to detect the number of fingers used by the subject. The number of fingers directs the path to robot that is to be followed. The camera placed on the robot capture the images of its surrounding, wherever it travels and send it back to the PC for monitoring. In this way, it can be used as a surveillance system. Experimental results show that the overall accuracy obtained above 90% for gesture recognition by which robot will be directed to follow the path. The system can be directly applied to defence grounds for detection of enemy, for spying purpose where the human reach is avoided or not recommended. This unit can be used for overcoming physical handicaps by helping in development of gesture-based wheel chairs, for control of home devices and appliances for persons with physical handicaps and/or elderly users with impaired mobility