In this project an interactive computer system is designed that envisions to contribute to young children's musical education. From literature, requirements for musical interaction were derived. In this paper these requirements and the design of the system are described

Dijk, E.M.A.G. van

Jansen, A.E.

Retra, J.

English

van Dijk, Elisabeth M.A.G.

University of Twente Research Information

Musical Multimodal Child Computer Interaction  Lisette Jansen, Betsy van Dijk Human Media Interaction, University of Twente  PO Box 217, 7500 AE Enschede, the Netherlands a.e.jansen@student.utwente.nl e.m.a.g.vandijk@ewi.utwente.nl +31 53 4893781  José Retra School of Education & Lifelong Learning University of Exeter Heavitree Road, Exeter, EX1 2LU, UK J.Retra@exeter.ac.ukABSTRACT In this project an interactive computer system is designed that envisions to contribute to young children's musical education. From literature, requirements for musical interaction were derived. In this paper these requirements and the design of the system are described. Author Keywords Multimodal interaction, young children, embodied agents, musical interaction, edutainment. ACM Classification Keywords H.5.2 [Information Interfaces and Presentation]: User Interfaces - input devices and strategies, interaction styles; H.5.5 [Information Interfaces and Presentation]: Sound and Music Computing – systems; K.3.1 [Computers and Education]: Computer Uses in Education. INTRODUCTION Musical skills can give people, both children and adults, a lot of joy during their lives. There are also indications that musical development helps children with general development [6]. These are reasons why it is worth making an effort to try to stimulate children’s musical abilities. In this project an interactive computer system is designed that envisions to contribute to young children's musical education. Children in early childhood have been chosen as the intended user group, because all children are born with musical abilities and early childhood is the critical period for learning music [6].    The objective of this research project is the design of an interactive music system for two and three-year-old children, with the intent to stimulate the child's inherent musical abilities through the interaction with the system. The system is preferably to be used in a home environment by one child at a time, without the help from adults or peers. The aim is that musical interaction between the child and the system will take place. The system should present its musical output in a way that evokes musical reactions of the child. Musical reactions are movements and sounds made by the child, like singing, dancing, stamping and clapping with the hands or clapping on the body. Multiple input modalities are used to detect the musical reactions of the child. The system adjusts its output according to the detected musical reactions. By making the child an active participant in the musical interaction, the system will hopefully contribute to the child's musical development.  Literature from the areas general child development, child-computer interaction and pre-school music education has been used to answer the following research questions:  • What are the requirements for an interactive edutainment system for two and three-year-old children?  • What musical interaction between child and adults stimulates the child's musical abilities? • What interaction styles are eligible for a natural musical interaction between the child and the system? Based on the answers to these questions, a system was designed. In the section ‘Interaction Design’ a description of the system is given. REQUIREMENTS FOR INTERACTIVE EDUTAINMENT This section describes the general abilities and characteristics of two and three-year-olds that were taken into consideration for the design of the system. In general, two-year-old children can walk and run. They can stand on tip-toes, jump with two feet together, take things apart and put them back together. Most two-year-olds can talk and sing. They can imitate parts of familiar songs and play with known songs and rhythms. They start to sing spontaneous songs. Two-year-olds are able to join in with action songs and respond through movement to different tempos in music. They enjoy clapping as an expression of rhythm. When children grow older, the physical coordination and fine-motor control increase. Also the vocal stamina and control increase. The spontaneous songs get longer. In general, three-year-olds can walk on tip-toes, stand on one foot and jump horizontally. A musical characteristic of three-year-olds is the singing of “pot-pourries”. These are spontaneous songs mixed up with parts of conventional songs, which may last up to several minutes. From a study of children’s needs and preferences from a child-computer interaction perspective, the following main issues were derived:    • Children need to feel in control of the technology they   Permission to make digital or hard copies oclassroom use is granted without fee provid  for profit or commercial advantage and thatcitation on the first page. To copy otherwiseredistribute to lists, requires prior specific p IDC '06, June 7-9, 2006 Tampere, Finland  Copyright 2006 ACM 1-59593-316-6/06/07f all or part of this work for personal or ed that copies are not made or distributed copies bear this notice and the full , to republish, to post on servers or to ermission and/or a fee. use and of their own learning [5] ... $5.00 163• Children enjoy technologies with multiple forms of interaction [1] • Children have a need to explore the world around them [2] • Fun and enjoyment support and deepen learning and facilitate engagement and motivation [3] INTERACTION STIMULATING MUSICAL ABILITIES The officially approved method for pre-school music education in the Netherlands is called “Music on the Lap” (MoL). The interaction between child and adult in this method has been chosen to be the example for the interaction between the child and the system. The issues exploration and enjoyment that were mentioned in the previous section are also important in MoL: MoL is process driven and enjoyment in the offered activities is important to motivate the children to join in [4].  MoL has a range of musical developmental goals [4]. For this project, three of these goals have been chosen: (1) sense of rhythm and timing, (2) sense of dynamics (loud and soft), (3) listening skills, which are in this project: learning to associate timbre with the corresponding musical instrument.  For young children music is a multimodal experience and singing and movement are natural musical activities of young children [6]. Movement is important in music education because all aspects of sound can be experienced through movement and by movement the child’s internal rhythm can be synchronised to an external input [4].  INTERACTION DESIGN Control, multiple forms of interaction including movement and singing, learning by exploration and process driven activities, and enjoyment were taken as the main requirements for the design of the system. A multimodal interactive system was designed that uses natural modalities of interaction and operates several communication channels in parallel. Natural interaction styles are necessary because children aged two and three cannot read yet. Moreover multimodal interaction is motivating. Figure 1: Screenshot of Panze The system consists of an input device connected to a television set. The television set shows an embodied agent, which is a cartoon-like kangaroo named Panze. The appearance and the name of Panze have been chosen by children in the target age group. Panze acts like a role model for the child; she motivates the child by singing, dancing and speaking. The aim is that the child will give musical reactions. The musical reactions of the child evoke a reaction in the program.  The input device also has the form of a kangaroo. The input kangaroo has a relatively large pouch in which tangible CDs and instruments can be inserted. These tangible objects are used for choosing songs and instruments. The background image of the television screen shows the chosen song and instruments. The activity of choosing instruments and hearing the corresponding sound is meant to learn the child to associate timbre with the corresponding musical instrument. The device contains several different sensors for detecting touch, movement, singing and the tangible objects. The system is turned on by touching the input kangaroo. Then the input kangaroo opens its eyes and it blinks its eyes every time the child touches it. The system turns off automatically when it detects no more reactions from the child for a few minutes. The input kangaroo then closes its eyes.  When Panze sings a song, she makes movements to the music. The first time the chosen song is played, Panze makes simple movements, like clapping or stamping in time to the music. Until the child reacts or ejects the “CD”, the song with these simple movements is repeated. If the child joins in the singing or the movements, it will be rewarded by visual and sound effects. Panze will start to vary the singing or the movements to give the child a new challenge. Thus movement detection is used for activities stimulating the child’s sense of rhythm and singing detection is used for activities stimulating the child’s sense of dynamics.   CONTINUING RESEARCH An interactive prototype of the system will be created. This prototype will be used to test how children react to Panze and the input kangaroo. The first results will be available at the IDC2006 conference. REFERENCES 1. Druin, A., Bederson, B., Boltman, A., Miura, A., Knotts-Callahan, D., Platt, M. 1999. ‘Children as Our Technology Design Partners’ In: Druin, A. (Ed.) 1999. The design of children's technology. San Francisco, CA: Morgan Kaufmann Publishers, 51-72. 2. Gopnik, A., Meltzoff, A. and Kuhl, P. 2001. How babies think. London:  Phoenix. 3. Price, S., Rogers, Y., Scaife, M., Stanton, D. and Neale, H. 2003. ‘Using ‘tangibles’ to promote novel forms of playful learning’ In: Interacting with computers, 15 (2), 169-185. 4. Retra, J.M. 2005. ‘Musical Movement Responses in Early Childhood Music Education Practice in The Netherlands’. 2nd Conference EXETER 2005, European Network for Music Educators and Researchers of Young Children (MERYC), University of Exeter, UK. 5. Siraj-Blatchford, J. and Whitebread, D. 2003. Supporting ICT in the early years. Berkshire: Open University Press. 6. Young, S. 2003. Music with the under-fours. London: RoutledgeFalmer.164

Musical Multimodal Child Computer Interaction

How babies think.

Music with the under-fours.

Musical Movement Responses in Early Childhood Music Education Practice

Supporting ICT in the early years. Berkshire:

The design of children's technology.

Using ‘tangibles’ to promote novel forms of playful learning’ In: Interacting with computers,

Musical Multimodal Child Computer Interaction

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information