Nonlinearity reduction of manifolds using Gaussian blur for handshape recognition based on multi-dimensional grids

This paper presents a hand-shape recognition algorithm based on using multi-dimensional grids (MDGs) to divide the feature space of a set of hand images. Principal Component Analysis (PCA) is used as a feature extraction and dimensionality reduction method to generate eigenspaces from example images. Images are blurred by convolving with a Gaussian kernel as a low pass filter. Image blurring is used to reduce the non-linearity in the manifolds within the eigenspaces where MDG structure can be used to divide the spaces linearly. The algorithm is invariant to linear transformations like rotation and translation. Computer generated images for different hand-shapes in Irish Sign Language are used in testing. Experimental results show accuracy and performance of the proposed algorithm in terms of blurring level and MDG size

Principal component pyramids using image blurring for nonlinearity reduction in hand shape recognition

The thesis presents four algorithms using a multistage hierarchical strategy for hand shape recognition. The proposed multistage hierarchy analyzes new patterns by projecting them into the different levels of a data pyramid, which consists of different principal component spaces. Image blurring is used to reduce the nonlinearity in manifolds generated by a set of example images. Flattening the space helps in classifying different hand shapes more accurately. Four algorithms using different pattern recognition techniques are proposed. The first algorithm is based on using perpendicular distance to measure the distance between new patterns and the nearest manifold. The second algorithm is based on using supervised multidimensional grids. The third algorithm uses unsupervised multidimensional grids to cluster the space into cells of similar objects. The fourth algorithm is based on training a set of simple architecture multi-layer neural networks at the different levels of the pyramid to map new patterns to the closest class. The proposed algorithms are categorized as example-based approaches where a large set of computer generated images are used to densely sample the space. Experimental results are presented to examine the accuracy and performance of the proposed algorithms. The effect of image blurring on reducing the nonlinearity in manifolds is examined. The results are compared with the exhaustive search scenario. The experimental results show that the proposed algorithms are applicable for real time applications with high accuracy measures. They can achieve frame rates of more than 10 frames per second and accuracies of up to 98% on test data

Handshape recognition using principal component analysis and convolutional neural networks applied to sign language

Handshape recognition is an important problem in computer vision with significant societal impact. However, it is not an easy task, since hands are naturally deformable objects. Handshape recognition contains open problems, such as low accuracy or low speed, and despite a large number of proposed approaches, no solution has been found to solve these open problems. In this thesis, a new image dataset for Irish Sign Language (ISL) recognition is introduced. A deeper study using only 2D images is presented on Principal Component Analysis (PCA) in two stages. A comparison between approaches that do not need features (known as end-to-end) and feature-based approaches is carried out. The dataset was collected by filming six human subjects performing ISL handshapes and movements. Frames from the videos were extracted. Afterwards the redundant images were filtered with an iterative image selection process that selects the images which keep the dataset diverse. The accuracy of PCA can be improved using blurred images and interpolation. Interpolation is only feasible with a small number of points. For this reason two-stage PCA is proposed. In other words, PCA is applied to another PCA space. This makes the interpolation possible and improves the accuracy in recognising a shape at a translation and rotation unknown in the training stage. Finally classification is done with two different approaches: (1) End-to-end approaches and (2) feature-based approaches. For (1) Convolutional Neural Networks (CNNs) and other classifiers are tested directly over raw pixels, whereas for (2) PCA is mostly used to extract features and again different algorithms are tested for classification. Finally, results are presented showing accuracy and speed for (1) and (2) and how blurring affects the accuracy