Vision-Based Three Dimensional Hand Interaction In Markerless Augmented Reality Environment

Kemunculan realiti tambahan membolehkan objek maya untuk wujud bersama dengan dunia sebenar dan ini memberi kaedah baru untuk berinteraksi dengan objek maya. Sistem realiti tambahan memerlukan penunjuk tertentu, seperti penanda untuk menentukan bagaimana objek maya wujud dalam dunia sebenar. Penunjuk tertentu mesti diperolehi untuk menggunakan sistem realiti tambahan, tetapi susah untuk seseorang mempunyai penunjuk tersebut pada bila-bila masa. Tangan manusia, yang merupakan sebahagian dari badan manusia dapat menyelesaikan masalah ini. Selain itu, tangan boleh digunakan untuk berinteraksi dengan objek maya dalam dunia realiti tambahan. Tesis ini membentangkan sebuah sistem realiti tambahan yang menggunakan tangan terbuka untuk pendaftaran objek maya dalam persekitaran sebenar dan membolehkan pengguna untuk menggunakan tangan yang satu lagi untuk berinteraksi dengan objek maya yang ditambahkan dalam tiga-matra. Untuk menggunakan tangan untuk pendaftaran dan interaksi dalam realiti tambahan, postur dan isyarat tangan pengguna perlu dikesan. The advent of augmented reality (AR) enables virtual objects to be superimposed on the real world and provides a new way to interact with the virtual objects. AR system requires an indicator to determine for how the virtual objects aligned in the real world. The indicator must first be obtained to access to a particular AR system. It may be inconvenient to have the indicator in reach at all time. Human hand, which is part of the human body may be a solution for this. Besides, hand is also a promising tool for interaction with virtual objects in AR environment. This thesis presents a markerless Augmented Reality system which utilizes outstretched hand for registration of virtual objects in the real environment and enables the users to have three dimensional (3D) interaction with the augmented virtual objects. To employ the hand for registration and interaction in AR, hand postures and gestures that the user perform has to be recognized

A fast and robust hand-driven 3D mouse

The development of new interaction paradigms requires a natural interaction. This means that people should be able to interact with technology with the same models used to interact with everyday real life, that is through gestures, expressions, voice. Following this idea, in this paper we propose a non intrusive vision based tracking system able to capture hand motion and simple hand gestures. The proposed device allows to use the hand as a "natural" 3D mouse, where the forefinger tip or the palm centre are used to identify a 3D marker and the hand gesture can be used to simulate the mouse buttons. The approach is based on a monoscopic tracking algorithm which is computationally fast and robust against noise and cluttered backgrounds. Two image streams are processed in parallel exploiting multi-core architectures, and their results are combined to obtain a constrained stereoscopic problem. The system has been implemented and thoroughly tested in an experimental environment where the 3D hand mouse has been used to interact with objects in a virtual reality application. We also provide results about the performances of the tracker, which demonstrate precision and robustness of the proposed syste

Visual-based fingertip detection for hand rehabilitation

This paper presents a visual detection of fingertips by using a classification technique based on the bag-of-words method. In this work, the fingertips are specifically of people who are holding a therapy ball, as it is intended to be used in a hand rehabilitation project. Speeded Up Robust Features (SURF) descriptors are used to generate feature vectors and then the bag-of-feature model is constructed by K-mean clustering which reduces the number of features. Finally, a Support Vector Machine (SVM) is trained to produce a classifier that distinguishes whether the feature vector belongs to a fingertip or not. A total of 4200 images, 2100 fingertip images and 2100 non-fingertip images, were used in the experiment. Our results show that the success rates for the fingertip detection are higher than 94% which demonstrates that the proposed method produces a promising result for fingertip detection for therapy-ball-holding hands. © 2018 Institute of Advanced Engineering and Science. All rights reserved

Exploring Circular Hough Transforms for Detecting Hand Feature Points in Noisy Images from Ghost-Circle Patterns

acceptedVersio

Two Hand Gesture Based 3D Navigation in Virtual Environments

Natural interaction is gaining popularity due to its simple, attractive, and realistic nature, which realizes direct Human Computer Interaction (HCI). In this paper, we presented a novel two hand gesture based interaction technique for 3 dimensional (3D) navigation in Virtual Environments (VEs). The system used computer vision techniques for the detection of hand gestures (colored thumbs) from real scene and performed different navigation (forward, backward, up, down, left, and right) tasks in the VE. The proposed technique also allow users to efficiently control speed during navigation. The proposed technique is implemented via a VE for experimental purposes. Forty (40) participants performed the experimental study. Experiments revealed that the proposed technique is feasible, easy to learn and use, having less cognitive load on users. Finally gesture recognition engines were used to assess the accuracy and performance of the proposed gestures. kNN achieved high accuracy rates (95.7%) as compared to SVM (95.3%). kNN also has high performance rates in terms of training time (3.16 secs) and prediction speed (6600 obs/sec) as compared to SVM with 6.40 secs and 2900 obs/sec

Multisensor Data Fusion Algorithm for Contactless 3D Position Measurement for Post-Stroke Hand Rehabilitation

Repetitive hand motion exercises help the patients regain their hand motor control. One of the widely used therapies of this type is the patient squeezing a flexible exercise ball in his/her hand repetitively. The exercise balls come at different levels of resistance to accommodate the different levels of limitation of the patients’ hands. However, one of the challenges is to measure objectively the progress that has been made without making any contact such that no additional weights loading the affected arm or hand of the patient. The presence of the exercise ball in the hand adds a degree of difficulty to the problem when an optical solution is adopted. This research attempted to investigate the enabler technology for contactless quantitative measurement system for monitoring the progress in such hand therapy. Evaluation of potential commercial-grade stereo-vision systems have been performed and fingertip detection algorithms have been proposed and evaluated. A total of 4200 images, 2100 fingertip images and 2100 non-fingertip images, were used in the experiment. Our results show that the success rates for the fingertip detection are higher than 94% which demonstrates that the proposed method produces a promising result for fingertip detection for therapy-ball-holding hands