THE USE OF CONTEXTUAL CLUES IN REDUCING FALSE POSITIVES IN AN EFFICIENT VISION-BASED HEAD GESTURE RECOGNITION SYSTEM

This thesis explores the use of head gesture recognition as an intuitive interface for computer interaction. This research presents a novel vision-based head gesture recognition system which utilizes contextual clues to reduce false positives. The system is used as a computer interface for answering dialog boxes. This work seeks to validate similar research, but focuses on using more efficient techniques using everyday hardware. A survey of image processing techniques for recognizing and tracking facial features is presented along with a comparison of several methods for tracking and identifying gestures over time. The design explains an efficient reusable head gesture recognition system using efficient lightweight algorithms to minimize resource utilization. The research conducted consists of a comparison between the base gesture recognition system and an optimized system that uses contextual clues to reduce false positives. The results confirm that simple contextual clues can lead to a significant reduction of false positives. The head gesture recognition system achieves an overall accuracy of 96% using contextual clues and significantly reduces false positives. In addition, the results from a usability study are presented showing that head gesture recognition is considered an intuitive interface and desirable above conventional input for answering dialog boxes. By providing the detailed design and architecture of a head gesture recognition system using efficient techniques and simple hardware, this thesis demonstrates the feasibility of implementing head gesture recognition as an intuitive form of interaction using preexisting infrastructure, and also provides evidence that such a system is desirable

Fingers micro-gesture recognition based on holoscopic 3D imaging system

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonMicro-gesture recognition has been widely research in recent years, in particular there has been a great focus on 3D micro-gesture recognition which consists of classifying the micro-gesture movements of the fingers for touch-less control applications. Holoscopic 3D imaging system mimics fly’s eye technique to capture true 3D scene which is enrich in both texture and motion information. As a result, holoscopic 3D imaging system shall be a suitable approach for robust recognition application. This PhD research focuses on innovative 3D micro-gesture recognition based on holoscopic 3D system which delivers robust and reliable performance with precision for 3D micro-gestures. Indeed this can be applied to other wide range of applications such as Internet of things (IoT), AR/VR, robotics and other touch-less interaction. Due to lack of holoscopic 3D dataset, a comprehensive 3D micro-gesture dataset (HoMG) includes both holoscopic 3D images and videos is prepared. It is a reasonable size holoscopic 3D dataset which is captured with different camera settings and conditions from 40 participants. Innovative 3D micro-gesture recognition is proposed based on 2D feature extraction methods with basic classification methods, the recognition accuracy can reach around 50.9%. For video-based data, the 3D feature extraction methods are achieved 66.7% recognition accuracy over 50.9% accuracy for micro-gesture images as the initial investigation. HoMG database held a challenge in IEEE International automatic face and gesture 2018, and 4 groups from the international research institutes joined the challenge and contributed many new methods as further development where the proposed method was published. The holoscopic 3D dataset further enrich innovative micro-gesture 3D recognition system is proposed and its performance is evaluated by carrying out like to like comparison with state of the art methods. In addition, a fast and efficient pre-processing algorithm for H3D images to extract the element images. Simplified viewpoint image extraction method are presented. A pre-trained CNN model with the attention mechanics is implemented based on VP image for the predicted probabilities of gesture. The proposed approached is further improved using voting strategy. The proposed approach achieves 87% accuracy, which outperform all existing state of the art methods on the image-based database. Advanced 3D micro-gesture recognition is investigated based on sequence video database, the end-to-end model has been used on effective H3D based micro-gesture recognition system. For front-end network, there are two method of traditional viewpoint image extraction and novel pseudo viewpoint image extraction have been used and evaluated. The pseudo viewpoint (PVP) front-end has been created, which used to deep learning networks understanding the implied 3D information of H3D imaging system. The viewpoint (VP) front-end follows the traditional H3D image method to extract and reconstruct the multi-viewpoint images. Both front-end have been feed in four popular advanced deep networks using for learning and classification. This experiments evaluated the performance of 2D/3D convolutional, mixing 2D and 3D convolutional and LSTM on the HoMG video database, which is beneficial to H3D imaging system using deep learning network. Finally, in order to obtain the high accuracies, the majority voting has been applied for further improve. The final results show that the performance is not only better than the traditional methods, but also superior to the existing deep learning based approaches, which clearly demonstrates the effectiveness of the proposed approach

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Synthesizing face sketches from real photos and its inverse have many applications. However, photo/sketch synthesis remains a challenging problem due to the fact that photo and sketch have different characteristics. In this work, we consider this task as an image-to-image translation problem and explore the recently popular generative models (GANs) to generate high-quality realistic photos from sketches and sketches from photos. Recent GAN-based methods have shown promising results on image-to-image translation problems and photo-to-sketch synthesis in particular, however, they are known to have limited abilities in generating high-resolution realistic images. To this end, we propose a novel synthesis framework called Photo-Sketch Synthesis using Multi-Adversarial Networks, (PS2-MAN) that iteratively generates low resolution to high resolution images in an adversarial way. The hidden layers of the generator are supervised to first generate lower resolution images followed by implicit refinement in the network to generate higher resolution images. Furthermore, since photo-sketch synthesis is a coupled/paired translation problem, we leverage the pair information using CycleGAN framework. Both Image Quality Assessment (IQA) and Photo-Sketch Matching experiments are conducted to demonstrate the superior performance of our framework in comparison to existing state-of-the-art solutions. Code available at: https://github.com/lidan1/PhotoSketchMAN.Comment: Accepted by 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)(Oral

Fair comparison of skin detection approaches on publicly available datasets

Skin detection is the process of discriminating skin and non-skin regions in a digital image and it is widely used in several applications ranging from hand gesture analysis to track body parts and face detection. Skin detection is a challenging problem which has drawn extensive attention from the research community, nevertheless a fair comparison among approaches is very difficult due to the lack of a common benchmark and a unified testing protocol. In this work, we investigate the most recent researches in this field and we propose a fair comparison among approaches using several different datasets. The major contributions of this work are an exhaustive literature review of skin color detection approaches, a framework to evaluate and combine different skin detector approaches, whose source code is made freely available for future research, and an extensive experimental comparison among several recent methods which have also been used to define an ensemble that works well in many different problems. Experiments are carried out in 10 different datasets including more than 10000 labelled images: experimental results confirm that the best method here proposed obtains a very good performance with respect to other stand-alone approaches, without requiring ad hoc parameter tuning. A MATLAB version of the framework for testing and of the methods proposed in this paper will be freely available from https://github.com/LorisNann