Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Music interaction: understanding music and human-computer interaction

We introduce and review recent research in Music and Human Computer Interaction, also known as Music Interaction. After a general overview of the discipline, we analyse the themes and issues raised by the fifteen chapters of this book, each of which presents recent research in this field. The bulk of this chapter is organised as an FAQ. This enables some FAQs to focus on cross cutting issues that appear in multiple chapters, and some chapters to feature in multiple FAQs. Broad topics include: the scope of research in Music Interaction; the role of HCI in Music Interaction; and conversely, the role of Music Interaction in HCI. High-level themes include embodied cognition, spatial cognition, evolutionary interaction, gesture, formal language, affective interaction, and methodologies from social science. Musical activities of interest include performance, composition, analysis, collaborative music making, and human and machine improvisation. Specific issues include: whether Music Interaction should be easy; what can be learned from the experience of being “in the groove”, and what can be learned from the deep commitment of musical amateurs. Broader issues include: what Music Interaction can offer traditional instruments and traditional musical activities; what relevance it has for non-musical domains; and ways in which Music Interaction can enable entirely new musical activities possible

Augmented reality meeting table: a novel multi-user interface for architectural design

Immersive virtual environments have received widespread attention as providing possible replacements for the media and systems that designers traditionally use, as well as, more generally, in providing support for collaborative work. Relatively little attention has been given to date however to the problem of how to merge immersive virtual environments into real world work settings, and so to add to the media at the disposal of the designer and the design team, rather than to replace it. In this paper we report on a research project in which optical see-through augmented reality displays have been developed together with prototype decision support software for architectural and urban design. We suggest that a critical characteristic of multi user augmented reality is its ability to generate visualisations from a first person perspective in which the scale of rendition of the design model follows many of the conventions that designers are used to. Different scales of model appear to allow designers to focus on different aspects of the design under consideration. Augmenting the scene with simulations of pedestrian movement appears to assist both in scale recognition, and in moving from a first person to a third person understanding of the design. This research project is funded by the European Commission IST program (IST-2000-28559)

Embodied cognition: A field guide

The nature of cognition is being re-considered. Instead of emphasizing formal operations on abstract symbols, the new approach foregrounds the fact that cognition is, rather, a situated activity, and suggests that thinking beings ought therefore be considered first and foremost as acting beings. The essay reviews recent work in Embodied Cognition, provides a concise guide to its principles, attitudes and goals, and identifies the physical grounding project as its central research focus

Feasibility study of an Integrated Program for Aerospace vehicle Design (IPAD). Volume 1A: Summary

IPAD was defined as a total system oriented to the product design process. This total system was designed to recognize the product design process, individuals and their design process tasks, and the computer-based IPAD System to aid product design. Principal elements of the IPAD System include the host computer and its interactive system software, new executive and data management software, and an open-ended IPAD library of technical programs to match the intended product design process. The basic goal of the IPAD total system is to increase the productivity of the product design organization. Increases in individual productivity were feasible through automation and computer support of routine information handling. Such proven automation can directly decrease cost and flowtime in the product design process

HyperCell: A Bio-inspired Design Framework for Real-time Interactive Architectures

This pioneering research focuses on Biomimetic Interactive Architecture using “Computation”, “Embodiment”, and “Biology” to generate an intimate embodied convergence to propose a novel rule-based design framework for creating organic architectures composed of swarm-based intelligent components. Furthermore, the research boldly claims that Interactive Architecture should emerge as the next truly Organic Architecture. As the world and society are dynamically changing, especially in this digital era, the research dares to challenge the Utilitas, Firmitas, and Venustas of the traditional architectural Weltanschauung, and rejects them by adopting the novel notion that architecture should be dynamic, fluid, and interactive. This project reflects a trajectory from the 1960’s with the advent of the avant-garde architectural design group, Archigram, and its numerous intriguing and pioneering visionary projects. Archigram’s non-standard, mobile, and interactive projects profoundly influenced a new generation of architects to explore the connection between technology and their architectural projects. This research continues this trend of exploring novel design thinking and the framework of Interactive Architecture by discovering the interrelationship amongst three major topics: “Computation”, “Embodiment”, and “Biology”. The project aims to elucidate pioneering research combining these three topics in one discourse: “Bio-inspired digital architectural design”. These three major topics will be introduced in this Summary.   “Computation”, is any type of calculation that includes both arithmetical and nonarithmetical steps and follows a well-defined model understood and described as, for example, an algorithm. But, in this research, refers to the use of data storage, parametric design application, and physical computing for developing informed architectural designs. “Form” has always been the most critical focus in architectural design, and this focus has also been a major driver behind the application computational design in Architecture. Nonetheless, this research will interpret the term “Form” in architecture as a continual “information processor” rather than the result of information processing. In other words, “Form” should not be perceived only as an expressive appearance based computational outcome but rather as a real-time process of information processing, akin to organic “Formation”. Architecture embodying kinetic ability for adjusting or changing its shape with the ability to process the surroundings and feedback in accordance with its free will with an inherent interactive intelligent movement of a living body. Additionally, it is also crucial to address the question of whether computational technologies are being properly harnessed, if they are only used for form-generating purposes in architecture design, or should this be replaced with real-time information communication and control systems to produce interactive architectures, with embodied computation abilities?   “Embodiment” in the context of this research is embedded in Umberto Eco’s vision on Semiotics, theories underlying media studies in Marshall McLuhan’s “Body Extension” (McLuhan, 1964), the contemporary philosophical thought of “Body Without Organs” (Gilles Deleuze and Félix Guattari, 1983), the computational Logic of ‘Swarm Behavior’ and the philosophical notion of “Monadology” proposed by Gottfried Leibniz (Leibniz, 1714). Embodied computation and design are predominant today within the wearable computing and smart living domains, which combine Virtual and Real worlds. Technical progress and prowess in VR development also contribute to advancing 3D smart architectural design and display solutions. The proposed ‘Organic body-like architectural spaces’ emphasize upon the realization of a body-like interactive space. Developing Interactive Architecture will imply eliciting the collective intelligence prevalent in nature and the virtual world of Big Data. Interactive Architecture shall thus embody integrated Information exchange protocols and decision-making systems in order to possess organic body-like qualities.   “Biology”, in this research explores biomimetic principles intended to create purposedriven kinetic and organic architecture. This involves a detailed study/critique of organic architecture, generating organic shapes, performance optimization based digital fabrication techniques and kinetic systems. A holistic bio-inspired architecture embodies multiple performance criteria akin to natural systems, which integrate structural, infrastructure performances throughout the growth of an organic body. Such a natural morphogenesis process of architectural design explores what Janine M. Benyus described as “learning the natural process”. Profoundly influenced by the processes behind morphogenesis, the research further explores Evolutionary Development Biology (Evo-Devo) explaining how embryological regulation strongly affect the resulting formations. Evo-Devo in interactive architecture implies the development of architecture based on three fundamental principles: “Simple to Complex”, “Geometric Information Distribution”, and “On/Off Switch and Trigger.” The research seeks to create a relatively intelligent architectural body, and the tactile interactive spatial environment by applying the extracted knowledge from the study of the aforementioned principles of Evo-Devo in the following fashion: A. Extract a Self-Similar Componential System based approach from the “Simple to Complex” principle of Evo-Devo B. Extract the idea of “Collective Intelligence” from “Geometric information Distribution” principle of Evo-Devo C. Extract the principle of “Assembly Regulation” from “On/Off switch and trigger” principle of Evo-Devo The “HyperCell” research, through an elaborate investigation on the three aforementioned topics, develops a design framework for developing real-time adaptive spatial systems. HyperCell does this, by developing a system of transformable cubic elements which can self-organize, adapt and interact in real-time. These Hypercells shall comprise an organic space which can adjust itself in relation to our human bodies. The furniture system is literally reified and embodied to develop an intra-active space that proactively provokes human movement. The space thus acquires an emotive dimension and can become your pet, partner, or even friend, and might also involve multiple usabilities of the same space. The research and its progression were also had actively connected with a 5-year collaborative European Culture project: “MetaBody”. The research thus involves exploration of Interactive Architecture from the following perspectives: architectural design, digital architectural history trajectory, computational technology, philosophical discourse related to the embodiment, media and digital culture, current VR and body-related technology, and Evolutionary Developmental Biology. “HyperCell” will encourage young architects to pursue interdisciplinary design initiatives via the fusion of computational design, embodiment, and biology for developing bio-inspired organic architectures

HyperCell: A Bio-inspired Design Framework for Real-time Interactive Architectures

This pioneering research focuses on Biomimetic Interactive Architecture using “Computationâ€, “Embodimentâ€, and “Biology†to generate an intimate embodied convergence to propose a novel rule-based design framework for creating organic architectures composed of swarm-based intelligent components. Furthermore, the research boldly claims that Interactive Architecture should emerge as the next truly Organic Architecture. As the world and society are dynamically changing, especially in this digital era, the research dares to challenge the Utilitas, Firmitas, and Venustas of the traditional architectural Weltanschauung, and rejects them by adopting the novel notion that architecture should be dynamic, fluid, and interactive. This project reflects a trajectory from the 1960’s with the advent of the avant-garde architectural design group, Archigram, and its numerous intriguing and pioneering visionary projects. Archigram’s non-standard, mobile, and interactive projects profoundly influenced a new generation of architects to explore the connection between technology and their architectural projects. This research continues this trend of exploring novel design thinking and the framework of Interactive Architecture by discovering the interrelationship amongst three major topics: “Computationâ€, “Embodimentâ€, and “Biologyâ€. The project aims to elucidate pioneering research combining these three topics in one discourse: “Bio-inspired digital architectural designâ€. These three major topics will be introduced in this Summary. “Computationâ€, is any type of calculation that includes both arithmetical and nonarithmetical steps and follows a well-defined model understood and described as, for example, an algorithm. But, in this research, refers to the use of data storage, parametric design application, and physical computing for developing informed architectural designs. “Form†has always been the most critical focus in architectural design, and this focus has also been a major driver behind the application computational design in Architecture. Nonetheless, this research will interpret the term “Form†in architecture as a continual “information processor†rather than the result of information processing. In other words, “Form†should not be perceived only as an expressive appearance based computational outcome but rather as a real-time process of information processing, akin to organic “Formationâ€. Architecture embodying kinetic ability for adjusting or changing its shape with the ability to process the surroundings and feedback in accordance with its free will with an inherent interactive intelligent movement of a living body. Additionally, it is also crucial to address the question of whether computational technologies are being properly harnessed, if they are only used for form-generating purposes in architecture design, or should this be replaced with real-time information communication and control systems to produce interactive architectures, with embodied computation abilities? “Embodiment†in the context of this research is embedded in Umberto Eco’s vision on Semiotics, theories underlying media studies in Marshall McLuhan’s “Body Extension†(McLuhan, 1964), the contemporary philosophical thought of “Body Without Organs†(Gilles Deleuze and Félix Guattari, 1983), the computational Logic of ‘Swarm Behavior’ and the philosophical notion of “Monadology†proposed by Gottfried Leibniz (Leibniz, 1714). Embodied computation and design are predominant today within the wearable computing and smart living domains, which combine Virtual and Real worlds. Technical progress and prowess in VR development also contribute to advancing 3D smart architectural design and display solutions. The proposed ‘Organic body-like architectural spaces’ emphasize upon the realization of a body-like interactive space. Developing Interactive Architecture will imply eliciting the collective intelligence prevalent in nature and the virtual world of Big Data. Interactive Architecture shall thus embody integrated Information exchange protocols and decision-making systems in order to possess organic body-like qualities. “Biologyâ€, in this research explores biomimetic principles intended to create purposedriven kinetic and organic architecture. This involves a detailed study/critique of organic architecture, generating organic shapes, performance optimization based digital fabrication techniques and kinetic systems. A holistic bio-inspired architecture embodies multiple performance criteria akin to natural systems, which integrate structural, infrastructure performances throughout the growth of an organic body. Such a natural morphogenesis process of architectural design explores what Janine M. Benyus described as “learning the natural processâ€. Profoundly influenced by the processes behind morphogenesis, the research further explores Evolutionary Development Biology (Evo-Devo) explaining how embryological regulation strongly affect the resulting formations. Evo-Devo in interactive architecture implies the development of architecture based on three fundamental principles: “Simple to Complexâ€, “Geometric Information Distributionâ€, and “On/Off Switch and Trigger.†The research seeks to create a relatively intelligent architectural body, and the tactile interactive spatial environment by applying the extracted knowledge from the study of the aforementioned principles of Evo-Devo in the following fashion: A. Extract a Self-Similar Componential Systembased approach from the “Simple to Complex†principle of Evo-Devo B. Extract the idea of “Collective Intelligence†from “Geometric information Distribution†principle of Evo-Devo C. Extract the principle of “Assembly Regulation†from “On/Off switch and trigger†principle of Evo-Devo The “HyperCell†research, through an elaborate investigation on the three aforementioned topics, develops a design framework for developing real-time adaptive spatial systems. HyperCell does this, by developing a system of transformable cubic elements which can self-organize, adapt and interact in real-time. These Hypercells shall comprise an organic space which can adjust itself in relation to our human bodies. The furniture system is literally reified and embodied to develop an intra-active space that proactively provokes human movement. The space thus acquires an emotive dimension and can become your pet, partner, or even friend, and might also involve multiple usabilities of the same space. The research and its progression were also had actively connected with a 5-year collaborative European Culture project: “MetaBodyâ€. The research thus involves exploration of Interactive Architecture from the following perspectives: architectural design, digital architectural history trajectory, computational technology, philosophical discourse related to the embodiment, media and digital culture, current VR and body-related technology, and Evolutionary Developmental Biology. “HyperCell†will encourage young architects to pursue interdisciplinary design initiatives via the fusion of computational design, embodiment, and biology for developing bio-inspired organic architectures

Design Strategies for Adaptive Social Composition: Collaborative Sound Environments

In order to develop successful collaborative music systems a variety of subtle interactions need to be identified and integrated. Gesture capture, motion tracking, real-time synthesis, environmental parameters and ubiquitous technologies can each be effectively used for developing innovative approaches to instrument design, sound installations, interactive music and generative systems. Current solutions tend to prioritise one or more of these approaches, refining a particular interface technology, software design or compositional approach developed for a specific composition, performer or installation environment. Within this diverse field a group of novel controllers, described as ‘Tangible Interfaces’ have been developed. These are intended for use by novices and in many cases follow a simple model of interaction controlling synthesis parameters through simple user actions. Other approaches offer sophisticated compositional frameworks, but many of these are idiosyncratic and highly personalised. As such they are difficult to engage with and ineffective for groups of novices. The objective of this research is to develop effective design strategies for implementing collaborative sound environments using key terms and vocabulary drawn from the available literature. This is articulated by combining an empathic design process with controlled sound perception and interaction experiments. The identified design strategies have been applied to the development of a new collaborative digital instrument. A range of technical and compositional approaches was considered to define this process, which can be described as Adaptive Social Composition. Dan Livingston