Benefits and pitfalls of multimedia and interactive features in technology-enhanced storybooks:A meta-analysis

A meta-analysis was conducted on the effects of technology-enhanced stories for young children’s literacy development when compared to listening to stories in more traditional settings like storybook reading. A small but significant additional benefit of technology was found for story comprehension (g+ = 0.17) and expressive vocabulary (g+ = 0.20), based on data from 2,147 children in 43 studies. When investigating the different characteristics of technology-enhanced stories, multimedia features like animated pictures, music, and sound effects were found beneficial. In contrast, interactive elements like hotspots, games, and dictionaries were found to be distracting. Especially for children disadvantaged because of less stimulating family environments, multimedia features were helpful and interactive features were detrimental. Findings are discussed from the perspective of cognitive processing theories

Visual melodies : design and evaluation of an interactive art installation for clinical environments

University of Technology, Sydney. Faculty of Arts and Social Sciences.Over the past decades there has been a growing recognition of the value of art activities, such as visual arts, music, drawing, dance, poetry and writing, as therapeutic approaches to enhance healthcare settings. Numerous studies have identified the value of art and design in decreasing depression and anxiety and creating a holistic healing environment for hospital visitors and patients, as well as offering a positive working environment for staff. Therefore, I propose that interactive multimedia art offers an important new therapeutic avenue as a service for engaging visitors, patients and staff in hospitals. Visual Melodies is an interactive art installation that engenders feelings of calm and relaxation in users. In this exegesis, I describe the theoretical background, development and evaluation of Visual Melodies. This creative trajectory draws on practice-based research, with the aims to create an interactive art installation, to evaluate its therapeutic potential, and to identify the semiotic dimensions of multimedia art that are most generally effective for producing therapeutic effects. Based on the literature of the different techniques used in art therapy, colour therapy and music therapy, I propose a bridge between these three therapies through a platform of an interactive multimedia installation – harnessing images, colours and sounds. Eight design principles that form the foundation of the practice were developed along with the discussion of the design elements that have been shown to be effective for enhancing relaxation. In line with the design principles, the design practice was then developed as a series of original landscape artworks and interactive animations accompanied by music specifically composed for the researcher. Audience feedback to the installation in a hospital waiting room was studied as a way of assessing its therapeutic potential. The evaluation feedback has been very positive and welcoming from visitors, patients and staff of all ages. Overall, Visual Melodies provides a relaxing and playful experience for the participants. The feelings most often reported were that of being relaxed, followed by calm, diverted, evoking memories and happy. This project demonstrates that it is beneficial to create a relaxing and supportive therapeutic interactive multimedia artwork for promoting holistic healing environments. The practice-based research and findings in this exegesis extend our understanding of how we can fuse artwork and technology, to transform our healthcare settings from sterile treatment spaces, into healing places where ‘care’ is built into the environment itself

Toward a model of computational attention based on expressive behavior: applications to cultural heritage scenarios

Our project goals consisted in the development of attention-based analysis of human expressive behavior and the implementation of real-time algorithm in EyesWeb XMI in order to improve naturalness of human-computer interaction and context-based monitoring of human behavior. To this aim, perceptual-model that mimic human attentional processes was developed for expressivity analysis and modeled by entropy. Museum scenarios were selected as an ecological test-bed to elaborate three experiments that focus on visitor profiling and visitors flow regulation

Sensing and mapping for interactive performance

This paper describes a trans-domain mapping (TDM) framework for translating meaningful activities from one creative domain onto another. The multi-disciplinary framework is designed to facilitate an intuitive and non-intrusive interactive multimedia performance interface that offers the users or performers real-time control of multimedia events using their physical movements. It is intended to be a highly dynamic real-time performance tool, sensing and tracking activities and changes, in order to provide interactive multimedia performances. From a straightforward definition of the TDM framework, this paper reports several implementations and multi-disciplinary collaborative projects using the proposed framework, including a motion and colour-sensitive system, a sensor-based system for triggering musical events, and a distributed multimedia server for audio mapping of a real-time face tracker, and discusses different aspects of mapping strategies in their context. Plausible future directions, developments and exploration with the proposed framework, including stage augmenta tion, virtual and augmented reality, which involve sensing and mapping of physical and non-physical changes onto multimedia control events, are discussed

Multiple Media Interfaces for Music Therapy

This article describes interfaces (and the supporting technological infrastructure) to create audiovisual instruments for use in music therapy. In considering how the multidimensional nature of sound requires multidimensional input control, we propose a model to help designers manage the complex mapping between input devices and multiple media software. We also itemize a research agenda