Hand gesture-based interactive puppetry system to assist storytelling for children

© 2016 The Author(s)Digital techniques have been used to assist narrative and storytelling, especially in many pedagogical practices. With the rapid development of HCI techniques, saturated with digital media in their daily lives, young children, demands more interactive learning methods and meaningful immersive learning experiences. In this paper, we propose a novel hand gesture-based puppetry storytelling system which provides a more intuitive and natural human computer interaction method for young children to develop narrative ability in virtual story world. Depth motion sensing and hand gestures control technology is utilized in the implementation of user-friendly interaction. Young players could intuitively use hand gestures to manipulate virtual puppet to perform story and interact with different items in virtual environment to assist narration. Based on the result of the evaluation, this novel digital storytelling system shows positive pedagogical functions on children’s narrating ability as well as the competencies of cognitive and motor coordination. The usability of the system is preliminary examined in our test, and the results which showed that young children can benefit from playing with Puppet Narrator

HAND GESTURE AND DETEKSI WAJAHDETECTION USING RASPBERRY PI

Face detection is currently used for various purposes, one of which is to record employees attendance. This strategy is ineffective since the employees still can hack the attendance by making their own photos and put them in their desks. If they are unable to come to the office,they can always ask their colleagues to submit their already available photos.Therefore, an alternative that can complement the current face detection method is highly needed.One of the methods that can be used is hand gesture detection.This study aims to detect hand gestures made by the employees to ensure whether they really come to work or not,so the chance for manipulation is quite small.For the purpose of hand gesture recognition, this study utilized Local Binary Pattern Histogram algorithm. LBPH is an algorithm used for the image matching process between images that have been given training and images taken in real time.The hand gesture image was first taken using a raspberry pi camera and then processed to examine whether it matches the registered ID or not.The results showed that ID recognition by using hand gestures is detectable and is in accordance with the registered ID.The number recognition in hand gestures includes numbers 1 to 10. The test results showed that, the average time required for reading hand gestures using a laptop was 9.2 seconds, while that of using raspberry was 14,2 seconds.Motion reading using a raspberry takes longer than that of using a laptop because the laptop's performance is higher than that of a raspberry

Hand features extractor using hand contour – a case study

Hand gesture recognition is an important topic in natural user interfaces (NUI). Hand features extraction is the first step for hand gesture recognition. This work proposes a novel real time method for hand features recognition. In our framework we use three cameras and the hand region is extracted with the background subtraction method. Features like arm angle and fingers positions are calculated using Y variations in the vertical contour image. Wrist detection is obtained by calculating the bigger distance from a base line and the hand contour, giving the main features for the hand gesture recognition. Experiments on our own data-set of about 1800 images show that our method performs well and is highly efficient

Raspberry Based Hand Gesture Recognition Using Haar Cascade and Local Binary Pattern Histogram

Many companies and even public institutions for civil servants currently use photo-taking for the attendance. However, this strategy is still considered ineffective since the employees still can hack the attendance by making their own photos and put them in their desks. Therefore, an alternative that can complement the current face detection method is highly needed so that the employee’s attendance can be directly monitored. One of the methods that can be used to detect the attendance is hand gesture detection. This research aims to detect hand gestures made by the employees to ensure whether they really come to work or not. This research make  the chance for manipulation using photo or fake GPS is quite small. For the purpose of hand gesture recognition, this study utilized Local Binary Pattern Histogram algorithm. The hand gesture image was first taken using a raspberry pi camera and then processed by the device to examine whether it matches the registered ID or not. The results showed that ID recognition by using hand gestures is detectable. The number recognition in hand gestures includes numbers 1 to 10. The test results showed that for 5 trials, the average time required for reading hand gestures using a laptop was 9.2 seconds, while that of using raspberry was 14.2 seconds. The results of this research show that the system has not been able to distinguish which hand is read first, so numbers that have the same number are considered the same, such as 81 and 18. So, the motion reading using a raspberry takes longer than that of using a laptop because the laptop's performance is higher than that of a raspberry and system cannot distinguish between numbers consisting of the same number

Two Hand Gesture Based 3D Navigation in Virtual Environments

Natural interaction is gaining popularity due to its simple, attractive, and realistic nature, which realizes direct Human Computer Interaction (HCI). In this paper, we presented a novel two hand gesture based interaction technique for 3 dimensional (3D) navigation in Virtual Environments (VEs). The system used computer vision techniques for the detection of hand gestures (colored thumbs) from real scene and performed different navigation (forward, backward, up, down, left, and right) tasks in the VE. The proposed technique also allow users to efficiently control speed during navigation. The proposed technique is implemented via a VE for experimental purposes. Forty (40) participants performed the experimental study. Experiments revealed that the proposed technique is feasible, easy to learn and use, having less cognitive load on users. Finally gesture recognition engines were used to assess the accuracy and performance of the proposed gestures. kNN achieved high accuracy rates (95.7%) as compared to SVM (95.3%). kNN also has high performance rates in terms of training time (3.16 secs) and prediction speed (6600 obs/sec) as compared to SVM with 6.40 secs and 2900 obs/sec

Improving the Working Memory During Early Childhood Education Through the Use of an Interactive Gesture Game-Based Learning Approach

One of the most socially and culturally advantageous uses of human–computer interaction is enhancing playing and learning for children. In this paper, gesture interactive game-based learning (GIGL) is tested to see if these kinds of applications are suitable to stimulate working memory (WM) and basic mathematical skills (BMS) in early childhood (5–6 years old) using a hand gesture recognition system. Hand gesture is being performed by the user and to control a computer system by that incoming information. The research was developed using a quasi-experimental design with a pre-test and post-test, using both an experimental and control group through three phases: the first one was the prior evaluation of the learner’s skills; a second phase in which the use of the technology was developed; and a final phase of evaluation. In the evaluation phases, working memory was measured using the Corsi task, and the basic mathematical skills using the test for the diagnosis of basic mathematical competencies (TEDI-MATH). The results provide clear evidence that the use of these technologies improved both working memory and basic mathematical skills. We can conclude that the children who used GIGL technology showed a significant increase in their learning performance in WM and BMS, surpassing those who did normal school activitiesS

La kinésica en la conversación coloquial del español de Bogotá

Doctorado en Ciencias del Lenguaje con mención en Lingüística AplicadaFil: Suárez Reina, Adriana Yamile. Universidad Nacional de Córdoba. Facultad de Lenguas; Argentina.La presente investigación aborda el estudio de los gestos en co-ocurrencia con el habla en las conversaciones coloquiales de un grupo de jóvenes universitarios de la ciudad de Bogotá. A partir de los estudios de McNeill (1985), se considera que los gestos no existen separadamente del lenguaje pues son una ventana al pensamiento. Aunque estos tienen diferentes sistemas de procesamiento, no son un accesorio del lenguaje, sino que hacen parte de un todo integrado facilitador de la comprensión del acto comunicativo. De acuerdo con ello, el objetivo fundamental del trabajo se orientó hacia la caracterización de los gestos en la conversación coloquial y la discusión de su naturaleza en la construcción del significado. La hipótesis consistió en que los gestos presentes en la conversación coloquial reflejan diversos modos de configurar el significado más allá de establecerse desde la generalización de repetir, contradecir, sustituir, complementar, acentuar o regular la comunicación verbal. Teniendo en cuenta esta mirada, se optó por un tipo de investigación mixto no experimental con un diseño transversal y de alcance descriptivo para identificar el papel de los gestos en el grupo estudiado. Los resultados evidenciaron que los gestos más usados por la población participante fueron los deícticos e icónicos. El primer tipo de gestos estaba en el plano de lo abstracto, razón por la cual los hablantes hacían uso de ellos a través de acciones corporizadas para hacer dichos términos más comunicables y comprensibles. El segundo grupo daba cuenta del proceso de pensamiento de los hablantes. Este grupo daba forma a cómo se estaba comprendiendo la idea a través de una experiencia de movimiento, es decir, la elaboración de pensamiento se interrelaciona con sus formas gestuales. Finalmente, se demuestra que los gestos presentes en la conversación coloquial de los jóvenes universitarios dan cuenta de la manera a través de la cual se configura el significado en tanto que es gracias a los gestos que las ideas abstractas pueden volverse físicas a través de la corporeización de información espacial, así como información temporal aportada a la representación esquemática de los conceptos usados.Fil: Suárez Reina, Adriana Yamile. Universidad Nacional de Córdoba. Facultad de Lenguas; Argentina