The study presents a set of interactive activities based on the dance and on the music for students with cognitive disabilities which are resisting to the therapies of rehabilitation with the physical therapist. The development of each one of these activities has achieved with teachers, physical therapist and students, all of them members of the research team. We have followed for this studio the principles of the Design Based Research. The main results are that the students have been more motivated in the accomplishment of activities of rehabilitation based on the music and on the dance. Also, these games have been motivating and very useful resource for the acceptance of conventional the therapies

Altanis

Chang

Chao

César Bernal Bravo

González-Ortega

Jose Antonio Piedra-Fernandez

Juan Jesus Ojeda-Castelo

Pyae

Sheu

Webster

Yuminaka

Procedia - Social and Behavioral Sciences

Bernal Bravo, César

Ojeda Castelo, Juan Jesús

Piedra Fernández, José Antonio

Repositorio de Patentes de la Universidad de Almería

Art activities with Kinect to students with cognitive disabilities: Improving all motor skills

Repositorio Institucional de la Universidad de Almería (Spain)

 Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Sciences 00 (2016) 000–000    www.elsevier.com/locate/procedia  1877-0428 © 2016 The Authors. Published by Elsevier Ltd. Peer-review under responsibility of the organizing committee of EDUHEM 2016.  7th International Conference on Intercultural Education “Education, Health and ICT for a Transcultural World”, EDUHEM 2016, 15-17 June 2016, Almeria, Spain Art activities with Kinect to students with cognitive disabilities: Improving all motor skills Cesar Bernal Bravob, Juan Jesus Ojeda-Casteloa, Jose Antonio Piedra-Fernandeza* aDepartment of Computer Science, University of Almería, Spain bDepartment of Education, University of Almería, Spain  Abstract The study presents a set of interactive activities based on the dance and on the music for students with cognitive disabilities which are resisting to the therapies of rehabilitation with the physical therapist. The development of each one of these activities has achieved with teachers, physical therapist and students, all of them members of the research team. We have followed for this studio the principles of the Design Based Research. The main results are that the students have been more motivated in the accomplishment of activities of rehabilitation based on the music and on the dance. Also, these games have been motivating and very useful resource for the acceptance of conventional the therapies. © 2016 The Authors. Published by Elsevier Ltd. Peer-review under responsibility of the organizing committee of EDUHEM 2016. Keywords: Kinect, cognitive disabilities, motor skills, physical rehabilititation, games. 1. Introduction The Kinect device is used in various fields of socio-educational intervention, for instance in the rehabilitation of motor skills (1) (2), on the accessibility of people with sensory disabilities in public spaces such as museums (3), and therapies associated with the leisure as for example for old people (4) as for young people with autism (5) with one of the most popular consoles on the market (XBOX) in all of their games, even in serious games developed specifically for this purpose.   * Corresponding author. Tel.: +3495015442. E-mail address: cbernal@ual.es 2 Author name / Procedia - Social and Behavioral Sciences 00 (2016) 000–000 On the other hand, expressive arts such as dance has been including in their programs: virtual resources and augmented reality (6), also devices like Microsoft Kinect for allowing the motion capture in real time by opening a range of expressive possibilities for representations (7). Studies on the combination of the body expression along with the characteristics of the Kinect device for education, presented new opportunities for learning (8), the improvement in motor skills and coordination (9), and the general improvement of learning such as memory and comprehension skills (10), (11). In particular, different studies show the benefits of physical activity with the use of the Kinect device for students with intellectual disabilities (12), the need to incorporate this type of devices for cognitive rehabilitation (13), and even the potential for the hands and facial expressions recognition (14).   2. Methodology From a group inter level and interdisciplinary work made up of teachers from the Special  Education Center Princesa Sofia and professors of the University of Almería, designed a set of activities related to the expression of artistic following principles of research-based design (15), (16). There have been two different cycles of research, the first in which all the members of the group are involved, have been put in common for discussion the lines of investigation of the stages of work with students with cognitive disabilities. The second cycle, have participated the faculty who directly works with the student and the student; this second cycle has been repeated 5 times as many as students participate and collaborate in research. We have collected record written and audio of all group sessions of both cycles, also were collected onto the second cycles of work sessions. The emerging categories of the first cycle has been instrumental and didactic training need in the use of the Kinect, the pedagogical position towards the integration of the device in the dynamics of teaching, the spatiality with and for students and its relationship with the Kinect.  The emerging categories of the 5 second cycles have been the realization and adaptation of the curriculum to a new medium, the times spaces of individual and collective work, changes in the student teacher interaction, personalization of the diversity of the student body activities and the training of students with the activities. The activities aimed at students and teachers of the Centre have been in 20 months.   3. Design For these activities, the Kinect technology has been used since it allowed a gesture recognition with full body, unlike other devices such as Leap Motion that allows recognition of gestures but only the hands. So far the Kinect sensor is the only tool that allows a recognition (tracking) of 20 joints. 3.1. Music The music section is composed of three activities see Fig. 1. These activities will be shown a 3D avatar, which mimics the movements that the users make on a virtual scene. The objective of these activities is that users play a musical instrument. Instruments that can be played are the cymbals and the piano. For each of the exercises the student must perform a characteristic gesture that represents the action of the instrument. In the exercise of the cymbals the user has first to extend his arms and then collapse them until your hands are at a very close distance. In the exercise of the piano, users have to lift your arms and then lower them until they make contact with the piano. When the gesture is successful activates a feedback, which in this case is the sound of cymbals or a musical note of the scale of the piano, depending on the chosen activity. The last activity is a joint activity where two users can interact at the same time each playing one of the musical instruments to try and compose a melody.   Author name / Procedia - Social and Behavioral Sciences 00 (2016) 000–000 3               3.2. Dance This section consists of two activities see Fig. 2. The aim is that students discover that through your body language can interact with the virtual world using a 3D avatar that will mimic their movements (just like in music exercises). In one of the activities it will sound an acoustic music to encourage the student to move. In this activity, there are three deployed gestures: raise arms, leave them in T pose or lower arms. When the user raise the arms falling flowers from the sky without stopping until the user place his arms in T pose that frozen mode is activated and the flowers will remain static on the stage or lower arms, which in this case the flowers disappear. Another developed exercise has as a goal to the user to work with a virtual object that will be placed in one of these four joints: right hand, left hand, right foot or left foot, at the option of the user. The physiotherapist of the Special Education Center makes some of the exercises of rehabilitation with a physical ball. In this activity students can do exercises with a virtual ball, following the same directions as the physiotherapist makes the daily sessions.  There is implemented two gestures in this exercise: throw the ball and lift an arm. When the user makes a similar gesture to the throwing a ball with the hand, i.e. first moved his arm back at the height of his head and immediately moved it forward at the same height. The action of this gesture on the stage are some fireworks which will appear the stage indicating to the user that has executed the movement correctly. However, if the user raises one of her arms the effect that will have on the exercise is that it restarts.            3.3. Painting The exercise of painting has several actions that can be performed: the thickness of the stroke, with what paint color and if you want to delete the drawing area. The colors that you can choose to paint are: green, red and blue, and the strokes are: fine, medium and thick. These options make that they can be carried out different activities depending on these attributes. An exercise is to draw in the middle a horizontal line and that they have to draw a line of different thickness over or under it. A variation is to draw two lines of different thickness and indicate to the user that you paint such a line below the line finer. In the activity of painting is also exercised laterality, to achieve this end draws a vertical color line in the middle of the drawing area and the user has to paint a line to the left or to the a b c Fig. 1. (a) Music activity with cymbals; (b) Music activity with piano (c) Music activity multiuser a b Fig. 2. (a) Dancing activity; (b) Dancing activity with virtual resources 4 Author name / Procedia - Social and Behavioral Sciences 00 (2016) 000–000 right of this. A variation of this exercise is done with several lines of different colors, and the student has to identify the color and make use of the concept of laterality. This activity can also be free exercise and students paint what want you at that moment. In this exercise, it is possible to first determine if you want to draw with his left hand or right hand, according to the student arm which is dominant, so the interaction as comfortable as possible for the user.   Acknowledgements This project has been supported by the Ministry of Education (TIN2013-41576-R). This work was funded by the EU ERDF and the Spanish Ministry of Economy and Competitiveness (MINECO) under Projects TIN2013-41576-R and the Andalusian Regional Government (Spain) under Project P10-TIC-6114. This work also received funding from the CEiA3 and CEIMAR consortiums. References 1. Capecci, M., Ceravolo, M. G., Ferracuti, F., Iarlori, S., Kyrki, V., Longhi, S., ... & Verdini, F. (2016, February). Physical rehabilitation exercises assessment based on Hidden Semi-Markov Model by Kinect v2. In 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI) (pp. 256-259). IEEE. 2. Yuminaka, Y., Mori, T., Watanabe, K., Hasegawa, M., & Shirakura, K. (2016). Non-contact Vital Sensing Systems Using a Motion Capture Device: Medical and Healthcare Applications. Key Engineering Materials, 698. 3. Zhu, Z., Khoo, W. L., Santistevan, C., Gosser, Y., Molina, E., Tang, H., ... & Tian, Y. (2016, March). EFRI-REM at CCNY: Research experience and mentoring for underrepresented groups in cross-disciplinary research on assistive technology. In 2016 IEEE Integrated STEM Education Conference (ISEC) (pp. 132-140). IEEE. 4. Pyae, A., Raitoharju, R., Luimula, M., Pitkäkangas, P., & Smed, J. (2016). Serious games and active healthy ageing: a pilot usability testing of existing games. International Journal of Networking and Virtual Organisations, 16(1), 103-120. 5. Bossavit, B., & Parsons, S. (2016, May). This is how I want to learn: High Functioning Autistic Teens Co-Designing a Serious Game. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 1294-1299). ACM. 6. Golz, P. & Shaw, A., 2014. Augmenting live performance dance through mobile technology. In BCS-HCI ’14 Proceedings of the 28th International BCS Human Computer Interaction Conference on HCI 2014. pp. 311–316 7. Golz, P. & Smith-Nunes, G., 2015. [arra]stre: A data driven ballet. In Electronic Visualisation and the Arts. pp. 90–91. 8. Sheu, F. R., & Chen, N. S. (2014). Taking a signal: a review of gesture-based computing research in education. Computers & Education, 78, 268-277 9. Altanis, G., Boloudakis, M., Retalis, S., & Nikou, N. (2013). Children with motor impairments play a Kinect learning game: first findings from a pilot case in an authentic classroom environment. Interaction Design and Architecture(s) Journal (IxD&A), 19, 91-104. 10. Chao, K. J., Huang, H. W., Fang, W. C., & Chen, N. S. (2013). Embodied play to learn: exploring Kinect-facilitated memory performance. British Journal of Educational Technology, 44(5), E151-E155. 11. Lu, S. J., Liu, Y. C., Chuang, Y. C., & Peng, C. P. (2012). A research of applying physically interactive games in the elementary Minan dialect curriculum and instruction. Curriculum & Instruction Quarterly, 15(2), 169-191. 12. Li, P. L., Wang, J. C., Wu, C. H., & Chen, C. H. (2014). The design of motion-sensing computer games applying on the physical education curriculum for  students with special needs. Special Education & Assistive Technology, 10, 41-51. 13. M.A. Musen, Y. Shahar, E.H. Shortliffe, Clinicaldecision-support systems, in: E.H. Shortliffe, L.E. Perreault,G. Wiederhold, L.M. Fagan (Eds.), Medical Informatics:Computer Applications in Health Care and Biomedicine,Springer, New York, NY, 2001, pp. 573–609. 14. González-Ortega, D., Díaz-Pernas, F.J., Martínez-Zarzuela, M., y Anton-Rodriguez, M. (2014). A kinect-based system for cognitive rehabilitation exercise monitoring. Computer Methods and Programs in Biomedicine, 113, 60-631. 15. Zheng, L. (2016). Reflection on Design-Based Research: Challenges and Future Direction. In State-of-the-Art and Future Directions of Smart Learning (pp. 293-296). Springer Singapore. 16. Juuti, K., Lavonen, J., & Meisalo, V. (2016). Pragmatic Design-based research–Designing as a shared activity of teachers and researches. In Iterative Design of Teaching-Learning Sequences (pp. 35-46). Springer Netherlands.  

Crossref

Art activities with Kinect to Students with Cognitive Disabilities: Improving all Motor Skills

Repositorio Institucional de la Universidad de Almería

http://repositorio.ual.es/bitstream/10835/5257/1/Art%20activities%20with%20Kinect%20to%20students%20with%20cognitive%20disabilities%3a%20Improving%20all%20motor%20skills.pdf

Art activities with Kinect to students with cognitive disabilities: Improving all motor skills

Abstract

Similar works

Full text

Available Versions

Repositorio de Patentes de la Universidad de Almería

Repositorio Institucional de la Universidad de Almería (Spain)

Crossref

Repositorio Institucional de la Universidad de Almería