Music Paint Machine: an interactive music system for educational and artistic purpose

This thesis investigates the porting process of the Music Paint Machine into the Kinect Music Paint Machine (KMPM) and examines the positive and negative aspects that were found using the brand-new device Microsoft Kinect in the artistic and educational field of interactive systems. A brief introduction of the Music Paint Machine, its component and its usage is initially outlined. Afterwards a further introduction about the Kinect is also necessary for understanding the components, and how they work. After a short explanation about drivers installation and driver utilized, the discussion then focuses on the analysis of the MAX/MSP patch created for realizing the Kinect Music Paint Machine. In the conclusion, pro and cons are outlined together with the future applications that the Kinect Music Paint Machine can hav

Web GIS in practice X: a Microsoft Kinect natural user interface for Google Earth navigation

This paper covers the use of depth sensors such as Microsoft Kinect and ASUS Xtion to provide a natural user interface (NUI) for controlling 3-D (three-dimensional) virtual globes such as Google Earth (including its Street View mode), Bing Maps 3D, and NASA World Wind. The paper introduces the Microsoft Kinect device, briefly describing how it works (the underlying technology by PrimeSense), as well as its market uptake and application potential beyond its original intended purpose as a home entertainment and video game controller. The different software drivers available for connecting the Kinect device to a PC (Personal Computer) are also covered, and their comparative pros and cons briefly discussed. We survey a number of approaches and application examples for controlling 3-D virtual globes using the Kinect sensor, then describe Kinoogle, a Kinect interface for natural interaction with Google Earth, developed by students at Texas A&M University. Readers interested in trying out the application on their own hardware can download a Zip archive (included with the manuscript as additional files 1, 2, &3) that contains a 'Kinnogle installation package for Windows PCs'. Finally, we discuss some usability aspects of Kinoogle and similar NUIs for controlling 3-D virtual globes (including possible future improvements), and propose a number of unique, practical 'use scenarios' where such NUIs could prove useful in navigating a 3-D virtual globe, compared to conventional mouse/3-D mouse and keyboard-based interfaces

Modelling gesture recognition systems

Gesture recognition technologies have recently become available to a general consumer market. The most notable of these technologies rely on camera to support gesture recognition using multiple sensors that allow for standard color pictures, infrared pictures and depth sensing. These capabilities allow for variety of applications, including a body's skeleton or hand recognition. The recognition features allow for a multiple type input control. They allow an actor, who is in front of the camera, to apply different hand gestures, or body movements as the controls for an application or game that is running on the system the camera is plugged into. This paper discusses modelling and simulation of gesture recognition technologies

Assistive Courseware for Dyslexic Children To Increase Learning Abilities Based on Kinect Technology (ABCDyslexic)

It is well known that every child whether it be in pre-school or primary school have the difficulties to start to learn about language. They do not recognize the alphabet thus making them to read or write even a word is hard enough. Teachers will teach them and assist them in this situation and usually those children will passed from this stage after some time. But there’s a medical conditions where some children have the difficulties to even recognize the shape of the alphabet and effecting in further progress of the studies about language. Those children faced a language-based learning disorder that interferes with the acquisition of print literacy which refers to reading, writing, and spelling. This learning disorder called as Dyslexia which refers to difficulties in reading, writing, and spelling. There is a lot of technique that created special to assists dyslexic children that mainly based on multisensory approach. It optimizes the usage of all sensory organ of a student in learning process. Previous method may not be fully utilized because of the limitation of the apparatus that needed in optimizing the use of sensory organ. So the proposed system is a courseware based on Kinect to help dyslexic to improve learning abilities was presented. The propose system will help dyslexic children to improve their learning abilities by using Kinect that will maximize the use of all sensory organ and based on the English syllabus for primary school in Standard 1 and Standard 2, that officially prepared by the Ministry of Education. The objective of this study is to identify and limitations of the current ways of teaching English to dyslexic pre-school children. Extend its purpose to the investigation of the use of gesture-based learning approach for teaching & learning to dyslexic pre-school children. Then, it purpose also to develop a courseware that based on Kinect and also to conduct user’s perception study towards the develop course. It focuses on dyslexic children at the age of 5-10 years old. During the development process, the incremental and Iterative methodology is chosen. All the analysis, design, and implementation phases performed at the same time thus producing a prototype on each phase. The cycle repeated continually until the final prototype successfully meets the requirements. At first the system should be develop using Microsoft Visual Studio C# and Kinect SDK, but after a few prototyping process, the system has been changed and develop using Adobe Flash. The Kinect application also unattached between the applications as it will allow user that don’t have Kinect to run the application. User may use Kinect or Mouse as a controller for the application.Mainly the system consists of 4 main modules. Each modules dedicated to train in certain area start from Module 1 continue till Module 4. Module 1 focus on getting to know the letters and numbers. Module 2 design to train children with proper spelling. The third module focuses on the self-introduction. Then the last module is for shapes recognition which will train children to differentiate between letters and numbers. After the final prototype has been produced, the user perception study was conducted. The study results helps to support the develop application for further improvement and alteration to the application. The study was conducted on 5 primary school situated around Kluang, Johor. The final result shows good feedback on the application towards them. The application ‘ABCDyslexic’ has the abilities to help dyslexic children to improve their learning abilities thus improving their learning in other aspect. The development of this application will surely contribute to them and also at the same time provide guidance for teacher in assisting Dyslexic children in today learning environment. But further development on this application should be done to improve it even further such as making it compatible in other platform and improving the current modules to really cater and help dyslexic children to improve their learning abilities

People tracking and following with a smart wheelchair using an omnidirectional camera and a RGB-D Camera

The project implements a new service that enables a smart wheelchair user and another person to have a normal talk while freely strolling around the environment, without the need of any interaction towards the wheelchair, called Jiaolong

Interação controlada por gestos em ambientes 3D distribuídos

Mestrado em Comunicação MultimédiaO ser humano naturalmente comunica através de gestos, expressões, palavras e movimentos que permitem descobrir o ambiente envolvente e manipular os objetos. A investigação e o desenvolvimento de novos dispositivos tecnológicos, como os sensores de movimento, permitem que atualmente se consiga adaptar cada vez mais os sistemas de forma a facilitar a interação entre o humano e o computador. As recentes pesquisas na área da interação humano-computador promoveram o desenvolvimento de interfaces naturais, que proporcionam formas mais intuitivas de interagir com aplicações multimédia interativas. De forma a tornar a interação com o computador mais natural é possível detetar os movimentos do utilizador através de um sensor de movimento e controlar os avatares através dos gestos. As interfaces 3D permitem ao utilizador ter um sentido de presença no ambiente e, com recurso a um conjunto de tecnologias atualmente disponíveis, é possível desenvolver um sistema de controlo de um avatar num ambiente virtual 3D. Do ponto de vista prático, esta investigação propõe um estudo de natureza exploratória, tendo como base a recolha bibliográfica nas áreas da interação humano-computador, interação natural, interfaces 3D, experiência do utilizador e usabilidade. O procedimento metodológico do estudo é investigação de desenvolvimento com a conceptualização e implementação de um protótipo de uma aplicação controlada por gestos num ambiente 3D colaborativo. A investigação através do desenvolvimento de um protótipo permitiu compreender as potencialidades e limitações das tecnologias que possibilitam a interação através de gestos num ambiente 3D multiutilizador e sistematizar um conjunto de recomendações para futuras aplicações deste tipo.The human being naturally communicates through gestures, expressions, words and movements that allow him to engage with the surrounding environment and manipulate objects. The research and development of new technological devices, such as motion capture sensors, make it so that nowadays, systems are increasingly adaptable, facilitating human-computer interaction. Recent studies in the human-computer interaction field have promoted natural user interface development, which generate more intuitive ways to approach interactive multimedia applications. In such a way as to make interacting with a computer more natural, it's possible to detect user movements through a motion sensor, and control avatars with such gestures. 3D interfaces allow the user to feel immersed in the environment and, resorting to a set of currently available technologies, it is possible to develop an avatar control system in a 3D virtual environment. From a practical standpoint, this research proposes an exploratory study, based on literature review on human-computer interaction, natural interaction, 3D interfaces, user experience and usability. This study's methodological procedure is based on developmental research, along with the conceptualization and implementation of a gesture-controlled, collaborative 3D environment application's prototype. Investigating through prototype development enabled the comprehension of potentialities and limitations of the technologies that allow multi-user 3D environment gesture interaction, as well as systematizing a set of recommendations for future applications of this type

Kinect and 3D GIS in Archaeology

Abstract -This paper explores the potential of using Microsoft's Kinect to create a low-cost and portable system to virtually navigate, through a prototype 3D GIS, the digitally reconstructed ancient Maya city and UNESCO World Heritage Site of Copan in Honduras. The 3D GIS, named QueryArch3D, was developed as part of the MayaArch3D project (http://mayaarch3d.unm.edu), which explores the possibilities of integrating databases and 3D digital tools for research and teaching on ancient architectures and landscapes. The developed system, based on the Flexible Action and Articulated Skeleton Toolkit (FAAST), controls in a remote and touchless mode the movements in the 3D environment in order to create a sense of spatial awareness and embodiment. A user can thus use gestures to interact with information stored in the spatial database, calling up photos, videos, textual descriptions as he/she moves through the virtual space of the ancient Maya city

Motion Based Learning for Preschools Mathematics via Kinect (Counting Number)

Kids start learning numbers since they in preschools. Every kids have difficulties to start learn about mathematics. They do not recognize and might be difficult for them to remember the numbers at first. Teacher will help them to teach and assist them to solve this problem and it usually happens after some stages and time. The only problem is education system use currently is not helping much in increasing the speed of children's learning. With current education system that sometime bored and only use whiteboard as medium of teaching, kids cannot give their full commitment in class. In Malaysia, traditional and conventional teaching still widely being practiced. However, with the advancement of technology nowadays, it can help to overcome this problem. By using current technology, it can help to complement current education system. Education system can utilize the advancement of technology to overcome problem such as boring and not interactive learning. Example of technology that can be use is Kinect. Kinect is a device that can track human body to interact with an application. It can help make learning become active, interactive and at it also involving all body parts. The objective of introducing kinect in education is to develop a motion based learning application for preschools' mathematics. To achieve this objective, the author has decided to test the application for preschools kid age from 3 to 6 years old. It will focus on teaching simple numbers. Kinect in mathematic will cover on basic numbers among preschool kids. Since the time given to develop this project is very limited, the methodology chosen for this development is throwaway prototype methodology. By using throwaway prototyping methodology this project have improved and increase user involvement. Results showing that kids enjoy the kinect technology in mathematics and teachers have found one of interactive ways of teaching. From data collected and analyzed, results showed that kids enjoy this kinect technology in mathematics and at same time teachers have found one of interactive ways of teaching