Do-It-Yourself Single Camera 3D Pointer Input Device

We present a new algorithm for single camera 3D reconstruction, or 3D input for human-computer interfaces, based on precise tracking of an elongated object, such as a pen, having a pattern of colored bands. To configure the system, the user provides no more than one labelled image of a handmade pointer, measurements of its colored bands, and the camera's pinhole projection matrix. Other systems are of much higher cost and complexity, requiring combinations of multiple cameras, stereocameras, and pointers with sensors and lights. Instead of relying on information from multiple devices, we examine our single view more closely, integrating geometric and appearance constraints to robustly track the pointer in the presence of occlusion and distractor objects. By probing objects of known geometry with the pointer, we demonstrate acceptable accuracy of 3D localization.Comment: 8 pages, 6 figures, 2018 15th Conference on Computer and Robot Visio

Estimating 2D Upper Body Poses from Monocular Images

Automatic estimation and recognition of poses from video allows for a whole range of applications. The research described here is an important step towards automatic extraction of 3D poses. We describe our research to extract the 2D joint locations of the people in meeting videos. The key point of the research described here is that we generalize over variations in appearance of both people and scene. This results in a robust detection of 2D joint locations. For the detection of different limbs, we employ a number of limb locators. Each of these uses a different set of image features. We evaluate our work on two videos that have been recorded in the meeting context. Our results are promising, yielding an average error of approximately 3-5 cm per joint

Data association and occlusion handling for vision-based people tracking by mobile robots

This paper presents an approach for tracking multiple persons on a mobile robot with a combination of colour and thermal vision sensors, using several new techniques. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is incorporated into the tracker. The paper presents a comprehensive, quantitative evaluation of the whole system and its different components using several real world data sets

Improved Hand-Tracking Framework with a Recovery Mechanism

Abstract−Hand-tracking is fundamental to translating sign language to a spoken language. Accurate and reliable sign language translation depends on effective and accurate hand-tracking. This paper proposes an improved hand-tracking framework that includes a tracking recovery algorithm optimising a previous framework to better handle occlusion. It integrates the tracking recovery algorithm to improve the discrimination between hands and the tracking of hands. The framework was evaluated on 30 South African Sign Language phrases that use: a single hand; both hands without occlusion; and both hands with occlusion. Ten individuals in constrained and unconstrained environments performed the gestures. Overall, the proposed framework achieved an average success rate of 91.8% compared to an average success rate of 81.1% using the previous framework. The results show an improved tracking accuracy across all signs in constrained and unconstrained environments

Fair comparison of skin detection approaches on publicly available datasets

Skin detection is the process of discriminating skin and non-skin regions in a digital image and it is widely used in several applications ranging from hand gesture analysis to track body parts and face detection. Skin detection is a challenging problem which has drawn extensive attention from the research community, nevertheless a fair comparison among approaches is very difficult due to the lack of a common benchmark and a unified testing protocol. In this work, we investigate the most recent researches in this field and we propose a fair comparison among approaches using several different datasets. The major contributions of this work are an exhaustive literature review of skin color detection approaches, a framework to evaluate and combine different skin detector approaches, whose source code is made freely available for future research, and an extensive experimental comparison among several recent methods which have also been used to define an ensemble that works well in many different problems. Experiments are carried out in 10 different datasets including more than 10000 labelled images: experimental results confirm that the best method here proposed obtains a very good performance with respect to other stand-alone approaches, without requiring ad hoc parameter tuning. A MATLAB version of the framework for testing and of the methods proposed in this paper will be freely available from https://github.com/LorisNann

Tracking Skin-Colored Objects in Real-Time

We present a methodology for tracking multiple skin-colored objects in a monocular image sequence. The proposed approach encompasses a collection of techniques that allow the modeling, detection and temporal association of skincolored objects across image sequences. A non-parametric model of skin color is employed. Skin-colored objects are detected with a Bayesian classifier that is bootstrapped with a small set of training data and refined through an off-line iterative training procedure. By using on-line adaptation of skin-color probabilities the classifier is able to cope with considerable illumination changes. Tracking over time is achieved by a novel technique that can handle multiple objects simultaneously. Tracked objects may move in complex trajectories, occlude each other in the field of view of a possibly moving camera and vary in number over time. A prototype implementation of the developed system operates on 320x240 live video in real time (28Hz), running on a conventional Pentium IV processor. Representative experimental results from the application of this prototype to image sequences are also presented. 1

HGR-Net: A Fusion Network for Hand Gesture Segmentation and Recognition

We propose a two-stage convolutional neural network (CNN) architecture for robust recognition of hand gestures, called HGR-Net, where the first stage performs accurate semantic segmentation to determine hand regions, and the second stage identifies the gesture. The segmentation stage architecture is based on the combination of fully convolutional residual network and atrous spatial pyramid pooling. Although the segmentation sub-network is trained without depth information, it is particularly robust against challenges such as illumination variations and complex backgrounds. The recognition stage deploys a two-stream CNN, which fuses the information from the red-green-blue and segmented images by combining their deep representations in a fully connected layer before classification. Extensive experiments on public datasets show that our architecture achieves almost as good as state-of-the-art performance in segmentation and recognition of static hand gestures, at a fraction of training time, run time, and model size. Our method can operate at an average of 23 ms per frame