Hand posture recognition using modified Ensemble of Shape Functions and Global Radius-based Surface Descriptor

This paper presents an approach to recognition of static hand gestures based on data acquired from 3D cameras and point cloud descriptors: Ensemble of Shape Functions and Global Radius-based Surface Descriptor. We describe the recognition algorithm consisting of: hand segmentation, noise removal and downsampling of point cloud, dividing point cloud bounding box to cells, feature extraction and normalization, gesture classification. Modifications of the descriptors are proposed in order to increase hand posture recognition rates and decrease quantity of used features as well as computational cost of the algorithm. The experiments performed on four challenging datasets using cross-validation tests prove the usefulness of our approach

Image processing system based on similarity/dissimilarity measures to classify binary images from contour-based features

Image Processing Systems (IPS) try to solve tasks like image classification or segmentation based on its content. Many authors proposed a variety of techniques to tackle the image classification task. Plenty of methods address the performance of the IPS [1], as long as the influence of many external circumstances, such as illumination, rotation, and noise [2]. However, there is an increasing interest in classifying shapes from binary images (BI). Shape Classification (SC) from BI considers a segmented image as a sample (backgroundsegmentation [3]) and aims to identify objects based in its shape..

Robotic Cameraman for Augmented Reality based Broadcast and Demonstration

In recent years, a number of large enterprises have gradually begun to use vari-ous Augmented Reality technologies to prominently improve the audiences’ view oftheir products. Among them, the creation of an immersive virtual interactive scenethrough the projection has received extensive attention, and this technique refers toprojection SAR, which is short for projection spatial augmented reality. However,as the existing projection-SAR systems have immobility and limited working range,they have a huge difficulty to be accepted and used in human daily life. Therefore,this thesis research has proposed a technically feasible optimization scheme so thatit can be practically applied to AR broadcasting and demonstrations. Based on three main techniques required by state-of-art projection SAR applica-tions, this thesis has created a novel mobile projection SAR cameraman for ARbroadcasting and demonstration. Firstly, by combining the CNN scene parsingmodel and multiple contour extractors, the proposed contour extraction pipelinecan always detect the optimal contour information in non-HD or blurred images.This algorithm reduces the dependency on high quality visual sensors and solves theproblems of low contour extraction accuracy in motion blurred images. Secondly, aplane-based visual mapping algorithm is introduced to solve the difficulties of visualmapping in these low-texture scenarios. Finally, a complete process of designing theprojection SAR cameraman robot is introduced. This part has solved three mainproblems in mobile projection-SAR applications: (i) a new method for marking con-tour on projection model is proposed to replace the model rendering process. Bycombining contour features and geometric features, users can identify objects oncolourless model easily. (ii) a camera initial pose estimation method is developedbased on visual tracking algorithms, which can register the start pose of robot to thewhole scene in Unity3D. (iii) a novel data transmission approach is introduced to establishes a link between external robot and the robot in Unity3D simulation work-space. This makes the robotic cameraman can simulate its trajectory in Unity3D simulation work-space and project correct virtual content. Our proposed mobile projection SAR system has made outstanding contributionsto the academic value and practicality of the existing projection SAR technique. Itfirstly solves the problem of limited working range. When the system is running ina large indoor scene, it can follow the user and project dynamic interactive virtualcontent automatically instead of increasing the number of visual sensors. Then,it creates a more immersive experience for audience since it supports the user hasmore body gestures and richer virtual-real interactive plays. Lastly, a mobile systemdoes not require up-front frameworks and cheaper and has provided the public aninnovative choice for indoor broadcasting and exhibitions

Graph spectral domain features for static hand gesture recognition

The graph spectral processing is gaining increasing interest in the computer vision society because of its ability to characterize the shape. However, the graph spectral methods are usually high computational cost and one solution to simplify the problem is to automatically divide the graph into several sub-graphs. Therefore, we utilize a graph spectral domain feature representation based on the shape silhouette and we introduce a fully automatic divisive hierarchical clustering method based on the shape skeleton for static hand gesture recognition. In particular, we establish the ability of the Fiedler vector for partitioning 3D shapes. Several rules are applied to achieve a stable graph segmentation. The generated sub-graphs are used for matching purposes. Supporting results based on several datasets demonstrate the performance of the proposed method compared to the state-of-the-art methods by increment 0.3% and 3.8% for two datasets

Novel Muscle Monitoring by Radiomyography(RMG) and Application to Hand Gesture Recognition

Conventional electromyography (EMG) measures the continuous neural activity during muscle contraction, but lacks explicit quantification of the actual contraction. Mechanomyography (MMG) and accelerometers only measure body surface motion, while ultrasound, CT-scan and MRI are restricted to in-clinic snapshots. Here we propose a novel radiomyography (RMG) for continuous muscle actuation sensing that can be wearable and touchless, capturing both superficial and deep muscle groups. We verified RMG experimentally by a forearm wearable sensor for detailed hand gesture recognition. We first converted the radio sensing outputs to the time-frequency spectrogram, and then employed the vision transformer (ViT) deep learning network as the classification model, which can recognize 23 gestures with an average accuracy up to 99% on 8 subjects. By transfer learning, high adaptivity to user difference and sensor variation were achieved at an average accuracy up to 97%. We further demonstrated RMG to monitor eye and leg muscles and achieved high accuracy for eye movement and body postures tracking. RMG can be used with synchronous EMG to derive stimulation-actuation waveforms for many future applications in kinesiology, physiotherapy, rehabilitation, and human-machine interface