An Immersive Telepresence System using RGB-D Sensors and Head Mounted Display

We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to ef- fectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The highly interactive and flexible environment can capture and sustain more student attention than the traditional classroom setting and, thus, delivers a compelling experience to the students. Our main focus here is to investigate possible solutions for the system design and implementation and devise strategies for fast, efficient computation suitable for visual data processing and network transmission. We describe the technique and experiments in details and provide quantitative performance results, demonstrating our system can be run comfortably and reliably for different application scenarios. Our preliminary results are promising and demonstrate the potential for more compelling directions in cyberlearning.Comment: IEEE International Symposium on Multimedia 201

Robust 3D Action Recognition through Sampling Local Appearances and Global Distributions

3D action recognition has broad applications in human-computer interaction and intelligent surveillance. However, recognizing similar actions remains challenging since previous literature fails to capture motion and shape cues effectively from noisy depth data. In this paper, we propose a novel two-layer Bag-of-Visual-Words (BoVW) model, which suppresses the noise disturbances and jointly encodes both motion and shape cues. First, background clutter is removed by a background modeling method that is designed for depth data. Then, motion and shape cues are jointly used to generate robust and distinctive spatial-temporal interest points (STIPs): motion-based STIPs and shape-based STIPs. In the first layer of our model, a multi-scale 3D local steering kernel (M3DLSK) descriptor is proposed to describe local appearances of cuboids around motion-based STIPs. In the second layer, a spatial-temporal vector (STV) descriptor is proposed to describe the spatial-temporal distributions of shape-based STIPs. Using the Bag-of-Visual-Words (BoVW) model, motion and shape cues are combined to form a fused action representation. Our model performs favorably compared with common STIP detection and description methods. Thorough experiments verify that our model is effective in distinguishing similar actions and robust to background clutter, partial occlusions and pepper noise

Vision based 3D Gesture Tracking using Augmented Reality and Virtual Reality for Improved Learning Applications

3D gesture recognition and tracking based augmented reality and virtual reality have become a big interest of research because of advanced technology in smartphones. By interacting with 3D objects in augmented reality and virtual reality, users get better understanding of the subject matter where there have been requirements of customized hardware support and overall experimental performance needs to be satisfactory. This research investigates currently various vision based 3D gestural architectures for augmented reality and virtual reality. The core goal of this research is to present analysis on methods, frameworks followed by experimental performance on recognition and tracking of hand gestures and interaction with virtual objects in smartphones. This research categorized experimental evaluation for existing methods in three categories, i.e. hardware requirement, documentation before actual experiment and datasets. These categories are expected to ensure robust validation for practical usage of 3D gesture tracking based on augmented reality and virtual reality. Hardware set up includes types of gloves, fingerprint and types of sensors. Documentation includes classroom setup manuals, questionaries, recordings for improvement and stress test application. Last part of experimental section includes usage of various datasets by existing research. The overall comprehensive illustration of various methods, frameworks and experimental aspects can significantly contribute to 3D gesture recognition and tracking based augmented reality and virtual reality.Peer reviewe

Using a serious game to assess spatial memory in children and adults

Short-term spatial memory has traditionally been assessed using visual stimuli, but not auditory stimuli. In this paper, we design and test a serious game with auditory stimuli for assessing short-term spatial memory. The interaction is achieved by gestures (by raising your arms). The auditory stimuli are emitted by smart devices placed at different locations. A total of 70 participants (32 children and 38 adults) took part in the study. The outcomes obtained with our game were compared with traditional methods. The results indicated that the outcomes in the game for the adults were significantly greater than those obtained by the children. This result is consistent with the assumption that the ability of humans increases continuously during maturation. Correlations were found between our game and traditional methods, suggesting its validity for assessing spatial memory. The results indicate that both groups easily learn how to perform the task and are good at recalling the locations of sounds emitted from different positions. With regard to satisfaction with our game, the mean scores of the children were higher for nearly all of the questions. The mean scores for all of the questions, except one, were greater than 4 on a scale from 1 to 5. These results show the satisfaction of the participants with our game. The results suggest that our game promotes engagement and allows the assessment of spatial memory in an ecological way

Integrating the Kinect camera, gesture recognition and mobile devices for interactive discussion

Session H4CThe Microsoft Kinect camera is a revolutionary and useful depth camera giving new user experience of interactive gaming on the Xbox platform through gesture or motion detection. Besides the infrared-based depth camera, an array of built-in microphones for voice command is installed along the horizontal bar of the Kinect camera. As a result, there are increasing interests to apply the Kinect camera for various real-life applications including the control of squirt guns for outdoor swimming pools. In additional to the Kinect camera, mobile devices such as the smartphones readily integrated with motion sensors have been used for different real-time control tasks like the remote control of robots. In this project, we propose to integrate the Microsoft Kinect camera together with the smartphones as intelligent control for interactive discussion or presentation for the future e-learning system. To demonstrate the feasibility of our proposal, a prototype of our proposed gesture recognition and command specification software is built using the C# language on the MS.NET platform, and will be evaluated with a careful plan. Furthermore, there are many interesting directions for further investigation of our proposal. © 2012 IEEE.published_or_final_versio

Art and the Human Condition: Incorporating Visual Analysis of Artworks into a Undergraduate Pre-Medicine Curriculum

This thesis project presents a sample class session for the course, The Human Condition: An Arts Perpective (ARTH 361), which will be a part of the optional Medical Humanities minor for the pre-medicine students at Virginia Commonwealth University (VCU). Dr. Margaret Lindauer, Associate Professor of Art History at VCU has directed the development of this course. She oversaw the creation of the preliminary syllabus, which includes readings and assignments relevant to the course. The Tour presented in this thesis project provides a model for planning other class sessions, some of which will be developed by Museum Studies graduate students, some of whom might not have previous experience facilitating gallery tours. I assert that tours such as the one proposed in this project provide pre-medicine students with the opportunity to practice observational skills outside of the clinical setting. In looking at art, pre-medicine students approach the act of observation from a different angle and discussion about art objects often have valuable insight about the medical profession they are entering into. This thesis report accordingly offers brief summaries of research studies that were consulted while the aforementioned tour was developed. It also includes the author\u27s experiences working with a group of pre-nursing undergraduate student at VCU, and it includes a summary of the devloped tour as well as feedback provided by a VMFA Education Department staff member

New Method for Optimization of License Plate Recognition system with Use of Edge Detection and Connected Component

License Plate recognition plays an important role on the traffic monitoring and parking management systems. In this paper, a fast and real time method has been proposed which has an appropriate application to find tilt and poor quality plates. In the proposed method, at the beginning, the image is converted into binary mode using adaptive threshold. Then, by using some edge detection and morphology operations, plate number location has been specified. Finally, if the plat has tilt, its tilt is removed away. This method has been tested on another paper data set that has different images of the background, considering distance, and angel of view so that the correct extraction rate of plate reached at 98.66%.Comment: 3rd IEEE International Conference on Computer and Knowledge Engineering (ICCKE 2013), October 31 & November 1, 2013, Ferdowsi Universit Mashha