Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Simulating activities: Relating motives, deliberation, and attentive coordination

Activities are located behaviors, taking time, conceived as socially meaningful, and usually involving interaction with tools and the environment. In modeling human cognition as a form of problem solving (goal-directed search and operator sequencing), cognitive science researchers have not adequately studied “off-task” activities (e.g., waiting), non-intellectual motives (e.g., hunger), sustaining a goal state (e.g., playful interaction), and coupled perceptual-motor dynamics (e.g., following someone). These aspects of human behavior have been considered in bits and pieces in past research, identified as scripts, human factors, behavior settings, ensemble, flow experience, and situated action. More broadly, activity theory provides a comprehensive framework relating motives, goals, and operations. This paper ties these ideas together, using examples from work life in a Canadian High Arctic research station. The emphasis is on simulating human behavior as it naturally occurs, such that “working” is understood as an aspect of living. The result is a synthesis of previously unrelated analytic perspectives and a broader appreciation of the nature of human cognition. Simulating activities in this comprehensive way is useful for understanding work practice, promoting learning, and designing better tools, including human-robot systems

INTERACTIVE PHYSICAL DESIGN AND HAPTIC PLAYING OF VIRTUAL MUSICAL INSTRUMENTS

International audienceIn Computer Music, a practical approach of many Digital Musical Instruments is to separate the gestural input stage from the sound synthesis stage. While these instruments offer many creative possibilities, they present a strong rupture with traditional acoustic instruments, as the physical coupling between human and sound is broken. This coupling plays a crucial role for the expressive musical playing of acoustic instruments; we believe restoring it in a digital context is of equal importance for revealing the full expressive potential of digital instruments. This paper first presents haptic and physical modelling technologies for representing the mechano-acoustical instrumental situation in the context of DMIs. From these technologies, a prototype environment has been implemented for both designing virtual musical instruments and interacting with them via a force feedback device, able to preserve the energetic coherency of the musician-sound chain

A MODELLER-SIMULATOR FOR INSTRUMENTAL PLAYING OF VIRTUAL MUSICAL INSTRUMENTS

International audienceThis paper presents a musician-oriented modelling and simulation environment for designing physically modelled virtual instruments and interacting with them via a high performance haptic device. In particular, our system allows restoring the physical coupling between the user and the manipulated virtual instrument, a key factor for expressive playing of traditional acoustical instruments that is absent in the vast majority of computer-based musical systems. We first analyse the various uses of haptic devices in Computer Music, and introduce the various technologies involved in our system. We then present the modeller and simulation environments, and examples of musical virtual instruments created with this new environment

Experimental Approaches to the Composition of Interactive Video Game Music

This project explores experimental approaches and strategies to the composition of interactive music for the medium of video games. Whilst music in video games has not enjoyed the technological progress that other aspects of the software have received, budgets expand and incomes from releases grow. Music is now arguably less interactive than it was in the 1990’s, and whilst graphics occupy large amounts of resources and development time, audio does not garner the same attention. This portfolio develops strategies and audio engines, creating music using the techniques of aleatoric composition, real-time remixing of existing work, and generative synthesisers. The project created music for three ‘open-form’ games : an example of the racing genre (Kart Racing Pro); an arena-based first-person shooter (Counter-Strike : Source); and a real-time strategy title (0 A.D.). These games represent a cross-section of ‘sandbox’- type games on the market, as well as all being examples of games with open-ended or open-source code

Virtual orchestration: a film composer's creative practice

The advent of digital technologies has led to a major change in the process of film music composition; consequent developments in music technology have forced film composers to adapt to this change. Technological innovations such as digital audio workstations (DAWs) and virtual musical instruments have made possible the creation of virtual orchestras that are technologically capable of simulating the sound and behaviour of a traditional acoustic orchestra. This has had an effect on film music production and on the creative process of the professional film composer in a way that today, creating orchestral simulations or 'mock-ups' that imitate live orchestras (or smaller ensembles) has become a requirement in the film industry and thus an essential part of the film-scoring process. In the context of contemporary film music production, this thesis investigates how orchestral simulations are composed and created using computer music technology and virtual sample-based instruments. In asking 'how', the focus is on the film composer's activities and thought processes during this creative cycle, along with the nature of the interactive relationship between composer and music materials. This study aims to show the complexity of the film composer's creative practice and to advance understanding of how the use of computer music technology and orchestral sample libraries is influencing the compositional process and compositional outcome. To address these questions, a qualitative multiple case study methodology approach was chosen that included examination of the practice of seven professional film composers working in the field of feature film as the primary valid source of data. The exploration involved semi-structured interviews with composers, observations and analysis of their studio practice and inspection of their compositional tools. Taken as a whole, the evidence provided by this study is that the process of creating orchestral simulations is a process of film music composition during which professional film composers are creating orchestral sounds through the use of computers, digital sequencing, samplers and sample-based virtual acoustic instruments for the realisation of musical works. It is a process of using and manipulating recorded samples of real acoustic instruments to generate an expressive and convincing musical performance through sample-based orchestral simulation. A characteristic of this compositional practice is that it is a continuous process that proceeds in stages over time where all procedures can be applied repeatedly between stages. The process of creating orchestral simulations for the purpose of the film score is a multifaceted compositional activity involving a complex set of relationships among different compositional states of mind and compositional activities in which film composers experience music and interact with musical materials and media in various ways. This creative activity is a process involving a single person and a mixture of various compositional tools, the composer's skills and abilities brought into existence through a creative process that requires a thorough blend of art and craft to be demonstrated at all times

Automation of play:theorizing self-playing games and post-human ludic agents

This article offers a critical reflection on automation of play and its significance for the theoretical inquiries into digital games and play. Automation has become an ever more noticeable phenomenon in the domain of video games, expressed by self-playing game worlds, self-acting characters, and non-human agents traversing multiplayer spaces. On the following pages, the author explores various instances of automated non-human play and proposes a post-human theoretical lens, which may help to create a new framework for the understanding of videogames, renegotiate the current theories of interaction prevalent in game studies, and rethink the relationship between human players and digital games