Evaluating the Efficiency of CBAM-ResNet by using Malaysian Sign Language (BIM)

Malaysian Sign Language has been widely used by deaf-mutes in our nation for communication since created in 1998. However, most Malaysians do not understand this sign language and have difficulties in communicating with deaf-mutes. This had discouraged deafmutes and made them felt helpless when not being understood. Issues such as unfair treatment in education and work sometimes happened. As machine learning and computer vision domain developed, these technologies provided alternates in bridging the communication gap between deaf-mutes and others. This research introduced a competitive CNNs based neural network, namely CBAM-ResNet to prove its efficiency in Malaysian Sign language recognition. A dataset consists of 2071 videos for 19 dynamic signs was built to provide instances for network training. Two different experiments were conducted for static and dynamic signs using CBAM-2DResNet and CBAM-3DResNet respectively by two CBAM integration methods, which known as ‘Within blocks’ and ‘Before classifier’. Performance metrics such as accuracy, loss, precision, recall, F1-score, confusion matrix, and training time were taken to evaluate models’ efficiency. The results showed that all CBAM-ResNet models implemented had good performances in image and video recognition tasks, with recognition rates of over 90 % with little variation. CBAM-ResNet ‘Before Classifier’ is more efficient than ‘Within blocks’ models of CBAM-ResNet in terms of various performance metrics with shorter training time required. The best trained CBAM-2DResNet was chosen to build a realtime sign recognition application based on the image recognition technique. All experiment results indicated the CBAM-ResNet ‘Before classifier’ effectiveness in recognizing Malaysian Sign Language and it’s worth of future research

Evaluating the Efficiency of CBAM-Resnet Using Malaysian Sign Language

The deaf-mutes population is constantly feeling helpless when others do not understand them and vice versa. To fill this gap, this study implements a CNN-based neural network, Convolutional Based AttentionModule (CBAM), to recognise Malaysian Sign Language (MSL) in videos recognition. This study has created 2071 videos for 19 dynamic signs. Two different experiments were conducted for dynamic signs, using CBAM-3DResNet implementing ‘Within Blocks’ and ‘Before Classifier’ methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time were recorded to evaluate the models’ efficiency. Results showed that CBAM-ResNet models had good performances in videos recognition tasks, with recognition rates of over 90% with little variations. CBAMResNet ‘Before Classifier’ is more efficient than ‘Within Blocks’ models of CBAM-ResNet. All experiment results indicated the CBAM-ResNet ‘Before Classifier’ efficiency in recognising Malaysian Sign Language and its worth of future research

Development of deep learning neural network for ecological and medical images

Deep learning in computer vision and image processing has attracted attentions from various fields including ecology and medical image. Ecologists are interested in finding an effective model structure to classify different species. Tradition deep learning model use a convolutional neural network, such as LeNet, AlexNet, VGG models, residual neural network, and inception models, are first used on classifying bee wing and butterfly datasets. However, insufficient data sample and unbalanced samples in each class have caused a poor accuracy. To make improvement the test accuracy, data augmentation and transfer learning are applied. Recently developed deep learning framework based on mathematical morphology also shows its effective in shape representation, contour detection and image smoothing. The experimental results in the morphological neural network shows this type of deep learning model is also effective in ecology datasets and medical dataset. Compared with CNN, the MNN could achieve a similar or better result in the following datasets. The chest X-ray images are notoriously difficult to analyze for the radiologists due to their noisy nature. The existing models based on convolutional neural networks contain a giant number of parameters and thus require multi-advanced GPUs to deploy. In this research, the morphological neural networks are developed to classify chest X-ray images, including the Pneumonia Dataset and the COVID-19 Dataset. A novel structure, which can self-learn a morphological dilation or erosion, is proposed for determining the most suitable depth of the adaptive layer. Experimental results on the chest X-ray dataset and the COVID-19 dataset show that the proposed model achieves the highest classification rate as comparing against the existing models. More significant improvement is that the proposed model reduces around 97% computational parameters of the existing models. Automatic identification of pneumonia on medical images has attracted intensive studies recently. The model for detecting pneumonia requires both a precise classification model and a localization model. A joint-task joint learning model with shared parameters is proposed to combine the classification model and segmentation model. To accurately classify and localize pneumonia area. Experimental results using the massive dataset of Radiology Society of North America have confirmed the efficiency of showing a test mean interception over union (IoU) of 89.27% and a mean precision of area detection result of 58.45% in segmentation model. Then, two new models are proposed to improve the performance of the original joint-task learning model. Two new modules are developed to improve both classification and segmentation accuracies in the first model. These modules including an image preprocessing module and an attention module. In the second model, a novel design is used to combine both convolutional layers and morphological layers with an attention mechanism. Experimental results performed on the massive dataset of the Radiology Society of North America have confirmed its superiority over other existing methods. The classification test accuracy is improved from 0.89 to 0.95, and the segmentation model achieves an improved mean precision result from 0.58 to 0.78. Finally, two weakly-supervised learning methods: class-saliency map and grad-cam, are used to highlight corresponding pixels or areas which have significant influence on the classification model, such that the refined segmentation can focus on the correct areas with high confidence

Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network

(e deaf-mutes population always feels helpless when they are not understood by others and vice versa. (is is a big humanitarian problem and needs localised solution. To solve this problem, this study implements a convolutional neural network (CNN), convolutional-based attention module (CBAM) to recognise Malaysian Sign Language (MSL) from images. Two different experiments were conducted for MSL signs, using CBAM-2DResNet (2-Dimensional Residual Network) implementing “Within Blocks” and “Before Classifier” methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time are recorded to evaluate the models’ efficiency. (e experimental results showed that CBAM-ResNet models achieved a good performance in MSL signs recognition tasks, with accuracy rates of over 90% through a little of variations. (e CBAM-ResNet “Before Classifier” models are more efficient than “Within Blocks” CBAM-ResNet models. (us, the best trained model of CBAM-2DResNet is chosen to develop a real-time sign recognition system for translating from sign language to text and from text to sign language in an easy way of communication between deaf-mutes and other people. All experiment results indicated that the “Before Classifier” of CBAMResNet models is more efficient in recognising MSL and it is worth for future research

Enhancing Sign Language Recognition through Fusion of CNN Models

This study introduces a pioneering hybrid model designed for the recognition of sign language, with a specific focus on American Sign Language (ASL) and Indian Sign Language (ISL). Departing from traditional machine learning methods, the model ingeniously blends hand-crafted techniques with deep learning approaches to surmount inherent limitations. Notably, the hybrid model achieves an exceptional accuracy rate of 96% for ASL and 97% for ISL, surpassing the typical 90-93% accuracy rates of previous models. This breakthrough underscores the efficacy of combining predefined features and rules with neural networks. What sets this hybrid model apart is its versatility in recognizing both ASL and ISL signs, addressing the global variations in sign languages. The elevated accuracy levels make it a practical and accessible tool for the hearing-impaired community. This has significant implications for real-world applications, particularly in education, healthcare, and various contexts where improved communication between hearing-impaired individuals and others is paramount. The study represents a noteworthy stride in sign language recognition, presenting a hybrid model that excels in accurately identifying ASL and ISL signs, thereby contributing to the advancement of communication and inclusivity