An Immersive Telepresence System using RGB-D Sensors and Head Mounted Display

We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to ef- fectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The highly interactive and flexible environment can capture and sustain more student attention than the traditional classroom setting and, thus, delivers a compelling experience to the students. Our main focus here is to investigate possible solutions for the system design and implementation and devise strategies for fast, efficient computation suitable for visual data processing and network transmission. We describe the technique and experiments in details and provide quantitative performance results, demonstrating our system can be run comfortably and reliably for different application scenarios. Our preliminary results are promising and demonstrate the potential for more compelling directions in cyberlearning.Comment: IEEE International Symposium on Multimedia 201

Wheelchair-based game design for older adults

Few leisure activities are accessible to institutionalized older adults using wheelchairs; in consequence, they experience lower levels of perceived health than able-bodied peers. Video games have been shown to be an engaging leisure activity for older adults. In our work, we address the design of wheelchair-accessible motion-based games. We present KINECTWheels, a toolkit designed to integrate wheelchair movements into motion-based games, and Cupcake Heaven, a wheelchair-based video game designed for older adults using wheelchairs. Results of two studies show that KINECTWheels can be applied to make motion-based games wheelchair-accessible, and that wheelchair-based games engage older adults. Through the application of the wheelchair as an enabling technology in play, our work has the potential of encouraging older adults to develop a positive relationship with their wheelchair. Copyright 2013 ACM

From ‘hands up’ to ‘hands on’: harnessing the kinaesthetic potential of educational gaming

Traditional approaches to distance learning and the student learning journey have focused on closing the gap between the experience of off-campus students and their on-campus peers. While many initiatives have sought to embed a sense of community, create virtual learning environments and even build collaborative spaces for team-based assessment and presentations, they are limited by technological innovation in terms of the types of learning styles they support and develop. Mainstream gaming development – such as with the Xbox Kinect and Nintendo Wii – have a strong element of kinaesthetic learning from early attempts to simulate impact, recoil, velocity and other environmental factors to the more sophisticated movement-based games which create a sense of almost total immersion and allow untethered (in a technical sense) interaction with the games’ objects, characters and other players. Likewise, gamification of learning has become a critical focus for the engagement of learners and its commercialisation, especially through products such as the Wii Fit. As this technology matures, there are strong opportunities for universities to utilise gaming consoles to embed levels of kinaesthetic learning into the student experience – a learning style which has been largely neglected in the distance education sector. This paper will explore the potential impact of these technologies, to broadly imagine the possibilities for future innovation in higher education

Gesture Recognition System Application to early childhood education

One of the most socially and culturally advantageous uses of human-computer interaction is enhancing playing and learning for children. In this study, gesture interactive game-based learning (GIGL) is tested to see if these kinds of applications are suitable to stimulate working memory (WM) and basic mathematical skills (BMS) in early childhood (5-6 years old) using a hand gesture recognition system. Hand gesture is being performed by the user and to control a computer system by that incoming information. We can conclude that the children who used GIGL technology showed a significant increase in their learning performance in WM and BMS, surpassing those who did normal school activities

Navigation and interaction in a real-scale digital mock-up using natural language and user gesture

This paper tries to demonstrate a very new real-scale 3D system and sum up some firsthand and cutting edge results concerning multi-modal navigation and interaction interfaces. This work is part of the CALLISTO-SARI collaborative project. It aims at constructing an immersive room, developing a set of software tools and some navigation/interaction interfaces. Two sets of interfaces will be introduced here: 1) interaction devices, 2) natural language (speech processing) and user gesture. The survey on this system using subjective observation (Simulator Sickness Questionnaire, SSQ) and objective measurements (Center of Gravity, COG) shows that using natural languages and gesture-based interfaces induced less cyber-sickness comparing to device-based interfaces. Therefore, gesture-based is more efficient than device-based interfaces.FUI CALLISTO-SAR

GESTURE RECOGNITION FOR PENCAK SILAT TAPAK SUCI REAL-TIME ANIMATION

The main target in this research is a design of a virtual martial arts training system in real-time and as a tool in learning martial arts independently using genetic algorithm methods and dynamic time warping. In this paper, it is still in the initial stages, which is focused on taking data sets of martial arts warriors using 3D animation and the Kinect sensor cameras, there are 2 warriors x 8 moves x 596 cases/gesture = 9,536 cases. Gesture Recognition Studies are usually distinguished: body gesture and hand and arm gesture, head and face gesture, and, all three can be studied simultaneously in martial arts pencak silat, using martial arts stance detection with scoring methods. Silat movement data is recorded in the form of oni files using the OpenNI ™ (OFW) framework and BVH (Bio Vision Hierarchical) files as well as plug-in support software on Mocap devices. Responsiveness is a measure of time responding to interruptions, and is critical because the system must be able to meet the demand

Designing wheelchair-based movement games

People using wheelchairs have access to fewer sports and other physically stimulating leisure activities than nondisabled persons, and often lead sedentary lifestyles that negatively influence their health. While motion- based video games have demonstrated great potential of encouraging physical activity among nondisabled players, the accessibility of motion-based games is limited for persons with mobility disabilities, thus also limiting access to the potential health benefits of playing these games. In our work, we address this issue through the design of wheelchair-accessible motion-based game controls. We present KINECTWheels, a toolkit designed to integrate wheelchair movements into motion-based games. Building on the toolkit, we developed Cupcake Heaven, a wheelchair-based video game designed for older adults using wheelchairs, and we created Wheelchair Revolution, a motion-based dance game that is accessible to both persons using wheelchairs and nondisabled players. Evaluation results show that KINECTWheels can be applied to make motion-based games wheelchair-accessible, and that wheelchair-based games engage broad audiences in physically stimulating play. Through the application of the wheelchair as an enabling technology in games, our work has the potential of encouraging players of all ages to develop a positive relationship with their wheelchair

Avatar Kinect: Drama in the Virtual Classroom among L2 Learners of English

This study presents a qualitative approach to exploring classroom behaviour using dramaturgical analysis of student interactions in relation with, and as mediated through, a gesture-based gaming software among L2 learners of English at two international branch campuses in the Arabian Gulf where face-to-face interactions between unrelated members of the opposite sex are generally discouraged. We investigated whether Avatar Kinect might provide a safe way for young males and females to interact while discussing social issues in a composition course. Data were collected through personal observation and survey. Five key themes emerged from the study. First, some participants chose to perform at front stage and others chose to remain back stage. Second, front stage participants chose avatars with gender and skin colour similar to themselves. Third, all participants appeared to be engaged in the interactive role play processes and with one another. Fourth, front stage actors appeared to act without inhibition. Finally, all participants expressed frustration with technology shortcomings

MATHNECT: Mathematics Motion Based Learning for Primary School

Courseware is now widely used in the process of learning and teaching, particularly at the primary school level and in mathematics classes. Students become less interested in learning mathematics because it involves abstract and complex concepts. Difficulty learning math is a profound and acknowledged difficulty among students throughout the generations. The way of teaching and learning mathematics has remained relatively the same in our education system. Students are unable to provide their entire focus in class because of the current educational system, which is sometimes boring and relies solely on the whiteboard as a medium of instruction. However, with today's technological advancements, this problem may be overcome. It can be used to enhance the current educational system by utilizing current technology. One of them is Kinect, this device is a motion-tracking technology that allows users to interact with applications by moving their bodies. The main objective of this Mathnect is to develop a motion-based learning environment by utilizing Kinect technology as a learning aid to improve student concentration and attention in mathematics class. This project is developed using a methodology ADDIE. The methodology of analysis employed in this phase, which includes content analysis, should be focused on the compatibility of the content and the target users. The game will be focusing on teaching simple mathematics number, which it will cover on basic numbers among primary school students. Thus, by implementing this project in primary schools, our educational environment can be improved to produce new adaptive learning methods. This Mathnect is beneficial for educators, students, and parents to be used in the classroom and can also be used for self-based learning. As a result, students will not be bored in the classroom at any time