Theories of the development of human communication

This article considers evidence for innate motives for sharing rituals and symbols from animal semiotics, developmental neurobiology, physiology of prospective motor control, affective neuroscience and infant communication. Mastery of speech and language depends on polyrhythmic movements in narrative activities of many forms. Infants display intentional activity with feeling and sensitivity for the contingent reactions of other persons. Talk shares many of its generative powers with music and the other ‘imitative arts’. Its special adaptations concern the capacity to produce and learn an endless range of sounds to label discrete learned understandings, topics and projects of intended movement

Music Reading For Everybody: Notation for Laptop Music Making

The purpose of this dissertation is to document the creation of a new music notation called KeyMusic for the ASCII/QWERTY keyboard on any computer platform. This notation is designed to enable those without musical training the ability to participate in music making. Creation of the notation is traced from its conception to the application of HCI (Human Computer Interaction) usability criteria to establish its efficacy for users. All of the pieces notated in this format so far, are transcriptions from traditional folk song and arrangements of classical melodies. The development of a chromatic, polytonal, multitimbral laptop instrument which is designed to display the notation onscreen while the user plays the chosen part is also documented as a proof of concept instrument in Max

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD

Developing the personal narratives of children with complex communication needs associated with intellectual disabilities: what is the potential of Storysharing®?

Background: Sharing personal experience in narrative is challenging for individuals with intellectual disabilities. The aim was to investigate the potential of Storysharing® intervention. Materials and Methods: The study involved eleven pupil-educational supporter dyads at a special school. Storysharing® was implemented over a fifteen-week period. Personal narratives were captured on video pre- and post-intervention. The data were analysed for discourse and narrative. Results: Significant differences revealed a decline in ‘query-answer’ sequences and an increase in supporter use of ‘prompts’. Post-intervention there were fewer story episodes. Narrative structure showed gains in action sequences leading to climax, and in closing elements, indicating a more complete narrative. Conclusions: The Storysharing® intervention appears to be associated with changes to the dyadic, personal narratives illustrating its potential

TinkRBooks : tinkerable story elements for emergent literacy

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 92-102).Printed words are an abstract representation of concepts. Today parents teach children how to read by demonstrating how text is related to imagery. I present textual tinkerability, an idea for demonstrating reading by using multisensory gestures to expose and alter the text-graphic relationship within the story. Tinkerability allows readers to physically express words as they read, giving them some degree of control over the narrative. Two interactive storybooks called TinkRBooks demonstrate how tinkerability supports parent-child emergent literacy. Design guidelines were developed to showcase how tinkerability can be used for creating educationally meaningful interactivity. TinkRBooks allows parents to gesturally modify and discuss how text relates to concepts within a narrative. TinkRBooks allows children to actively explore the abstract relationship between printed words and their meanings, even before this relationship is properly understood. This ability to explore textual representation changes the way parents read to their children during emergent literacy. When using a TinkRBook, parents spend more time talking, discussing more comprehensive ideas with their children and provoking more meta dialogue than with regular books. TinkRBook also encourages children to drive their reading inquiry, by actively demonstrating the concepts relating to vocabulary schema within the narrative. The result is a new story sharing experience that benefits both parents and children by allowing them to understand how the choice of words impacts the story experience.by Angela Chang.Ph.D

Review: Development and technical design of tangible user interfaces in wide-field areas of application

A tangible user interface or TUI connects physical objects and digital interfaces. It is more interactive and interesting for users than a classic graphic user interface. This article presents a descriptive overview of TUI's real-world applications sorted into ten main application areas-teaching of traditional subjects, medicine and psychology, programming, database development, music and arts, modeling of 3D objects, modeling in architecture, literature and storytelling, adjustable TUI solutions, and commercial TUI smart toys. The paper focuses on TUI's technical solutions and a description of technical constructions that influences the applicability of TUIs in the real world. Based on the review, the technical concept was divided into two main approaches: the sensory technical concept and technology based on a computer vision algorithm. The sensory technical concept is processed to use wireless technology, sensors, and feedback possibilities in TUI applications. The image processing approach is processed to a marker and markerless approach for object recognition, the use of cameras, and the use of computer vision platforms for TUI applications.Web of Science2113art. no. 425

Musical linguistics: How music and artistic creativity when delivered as a linguistic practice, help students master academic skills in English language arts

This Masters project consists of two elements: 1) an integrated after-school program to improve student English language reading and academic outcomes for third graders\u27 vocabulary development by incorporating music, artistic creativity and linguistics; 2) a pilot sample curriculum that demonstrates the approach for building student comprehension through musical theater and Science, Technology, Engineering, and Mathematics (STEM) content experiences. Called Water Buddy , this is an after-school program uses singing, dancing, writing, and play to build reading and vocabulary skills. The goal is to improve learner academic outcomes by mastering the elemental building blocks of words, letters, symbols by making memorable the sound units, idioms, print conventions, and concepts that they were previously lacking