Hand gesture recognition based on fusion of moments

This work is focussed on three main issues in developing a gesture recognition system. These are (i) Threshold independent skin colour segmentation using Modified K-means clustering and Mahalanobish distance (ii) illumination normalization (iii) user independent gesture recognition based on fusion of Moments. Since skin pixels can vary with different illumination condition, to find the range of skin pixels, becomes a hard task in case of colour space based skin colour segmentation. This work proposes a semi-supervised learning algorithm based on modified K-means clustering and Mahalanobis distance to extract human skin colour regions from the static hand gesture colour images. An efficient illumination invariant algorithm based on power law transform and averaging RGB colour space is proposed. Normalized binary silhouette is extracted from the hand gesture image and background and object noise is removed by Morphological filtering. Non-orthogonal moments like geometric moments and orthogonal moments like Tchebichef and Krawtchouk moments are used here as features. The Krawtchouk moment features are found to be very effective in hand gesture recognition compared to Tchebichef and Geometric moment features. To make the system real time efficient, different users are used for training and testing. In user-independent situation, neither of these moments has shown efficient classification accuracy. To improve the performance of classification, two feature fusion strategies have been proposed in this work; serial feature fusion and parallel feature fusion. A feed-forward multi-layer perceptron (MLP) based artificial neural network classifier is used in this work as a classifier. The proposed two fusion based moment features especially parallel fusion of Krawtchouk and Tchebichef moment has shown better performance as user-independent. The proposed hand gesture recognition system can be well realized for real time implementation of gesture based applications

Efficient labeling of solar flux evolution videos by a deep learning model

Machine learning (ML) is becoming a critical tool for interrogation of large complex data. Labeling, defined as the process of adding meaningful annotations, is a crucial step of supervised ML. However, labeling datasets is time consuming. Here we show that convolutional neural networks (CNNs), trained on crudely labeled astronomical videos, can be leveraged to improve the quality of data labeling and reduce the need for human intervention. We use videos of the solar magnetic field, crudely labeled into two classes: emergence or non-emergence of bipolar magnetic regions (BMRs), based on their first detection on the solar disk. We train CNNs using crude labels, manually verify, correct labeling vs. CNN disagreements, and repeat this process until convergence. Traditionally, flux emergence labelling is done manually. We find that a high-quality labeled dataset, derived through this iterative process, reduces the necessary manual verification by 50%. Furthermore, by gradually masking the videos and looking for maximum change in CNN inference, we locate BMR emergence time without retraining the CNN. This demonstrates the versatility of CNNs for simplifying the challenging task of labeling complex dynamic events.Comment: 16 pages, 7 figures, published in Nature Astronomy, June 27, 202

Deep Learning based Skin-layer Segmentation for Characterizing Cutaneous Wounds from Optical Coherence Tomography Images

Optical coherence tomography (OCT) is a medical imaging modality that allows us to probe deeper substructures of skin. The state-of-the-art wound care prediction and monitoring methods are based on visual evaluation and focus on surface information. However, research studies have shown that sub-surface information of the wound is critical for understanding the wound healing progression. This work demonstrated the use of OCT as an effective imaging tool for objective and non-invasive assessments of wound severity, the potential for healing, and healing progress by measuring the optical characteristics of skin components. We have demonstrated the efficacy of OCT in studying wound healing progress in vivo small animal models. Automated analysis of OCT datasets poses multiple challenges, such as limitations in the training dataset size, variation in data distribution induced by uncertainties in sample quality and experiment conditions. We have employed a U-Net-based model for the segmentation of skin layers based on OCT images and to study epithelial and regenerated tissue thickness wound closure dynamics and thus quantify the progression of wound healing. In the experimental evaluation of the OCT skin image datasets, we achieved the objective of skin layer segmentation with an average intersection over union (IOU) of 0.9234. The results have been corroborated using gold-standard histology images and co-validated using inputs from pathologists. Clinical Relevance: To monitor wound healing progression without disrupting the healing procedure by superficial, noninvasive means via the identification of pixel characteristics of individual layers.Comment: Accepte

Homogenising SoHO/EIT and SDO/AIA 171\AA ~ Images: A Deep Learning Approach

Extreme Ultraviolet images of the Sun are becoming an integral part of space weather prediction tasks. However, having different surveys requires the development of instrument-specific prediction algorithms. As an alternative, it is possible to combine multiple surveys to create a homogeneous dataset. In this study, we utilize the temporal overlap of SoHO/EIT and SDO/AIA 171~\AA ~surveys to train an ensemble of deep learning models for creating a single homogeneous survey of EUV images for 2 solar cycles. Prior applications of deep learning have focused on validating the homogeneity of the output while overlooking the systematic estimation of uncertainty. We use an approach called `Approximate Bayesian Ensembling' to generate an ensemble of models whose uncertainty mimics that of a fully Bayesian neural network at a fraction of the cost. We find that ensemble uncertainty goes down as the training set size increases. Additionally, we show that the model ensemble adds immense value to the prediction by showing higher uncertainty in test data that are not well represented in the training data.Comment: 20 pages, 8 figures, accepted for publication in ApJ