Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

WEARABLE PRIVACY PROTECTION WITH VISUAL BUBBLE

Wearable cameras are increasingly used in many different applications such as entertainment, security, law enforcement and healthcare. In this thesis, we focus on the application of the police worn body camera and behavioral recording using a wearable camera for one-on-one therapy with a child in a classroom or clinic. To protect the privacy of other individuals in the same environment, we introduce a new visual privacy protection technique called visual bubble. Visual bubble is a virtual zone centered around the camera for observation whereas the rest of the environment and people are obfuscated. In contrast to most existing visual privacy protection systems that rely on visual classifiers, visual bubble is based on depth estimation to determine the extent of privacy protection. To demonstrate this concept, we construct a wearable stereo camera for depth estimation on the Raspberry Pi platform. We also propose a novel framework to quantify the uncertainty in depth measurements so as to minimize a statistical privacy risk in constructing the depth-based privacy bubble. To evaluate our system, we have collected three datasets. The effectiveness of the proposed scheme is demonstrated with experimental results

SuperFight I: the battle to understand a space through the behaviour of its occupants

This dissertation describes an interaction design project which investigates whether the symbiosis of physical and digital environments might be used to create a stronger sense of ‘place’ for the occupants of the physical space. Sensing technology, implicit interaction, ambient interfaces, game strategy and a network connection were combined in an attempt to increase participants' awareness of their physical actions and location. The principles and theory underpinning this project are discussed, after which a list of criteria for an interactive system designed for public spaces is drawn up. The design of SuperFight I is described and evaluated in relation to this theoretical background. Finally suggestions are made for future areas of research that might be undertaken in order to develop the system further

Designing performance systems for audience inclusion

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 154-168).We define the concept of the Hyperaudience and a unique approach towards designing real-time interactive performance systems: the design of these systems encourages audience participation and augments the experience of audience members through interconnected networks. In doing so, it embraces concepts found in ubiquitous computing, affective computing, interactive arts, music, theatrical tradition, and pervasive gaming. In addition, five new systems are demonstrated to develop a framework for thinking about audience participation and orchestrating social co-presence in and beyond the performance space. Finally, the principles and challenges that shaped the design of these five systems are defined by measuring, comparing, and evaluating their expressiveness and communicability.by Akito Van Troyer.S.M

Dervish Sound Dress; An Investigation of Wearable Technology Using Computer Music and Haptic Mechanisms for Live Performance

The realm of this thesis combines the areas of computer music, fashion design, digital art, smart clothing, biometrics, cultural traditions and performance. Dervish Sound Dress is a wearable piece of technology; a garment that is inspired by the sacred ‘turning’ experience of the Whirling Dervishes or the Mevlevi Sufi order in Turkey known as the sema. It utilizes the fundamental aspects of the sema such as music, performance and body movement through spiritual elation by creating a unique and interactive experience. Wearable technology is a burgeoning field of research. Fashion designers who are using smart textiles or integrating fashion and technology in some way require collaboration with electrical engineers and programming professionals. The garment functions as a body instrument and can be manipulated by the wearer. The cultural traditions of the Mevlevi Sufis and their metaphysical experience during the turning ritual of the sema performance is the inspiration behind the creation of a garment that emulates sounds by using body movement. Dervish Sound Dress is outfitted with sensors that trigger musical sounds when the wearer touches the bodice interface or changes gesture or movement. The wearer is alerted to the sounds through the use of haptics that are sensed on the body. The sensation is similar to when a musician plays an instrument that reverberates resulting in an immersive relationship that goes further than the auditory. The aim is to develop garments that will inspire the creation of musical sounds that can be controlled by an intuitive interface in clothing. It is a study that uses technology and performance by taking a sacred experience and creating artistic expression. Dervish Sound Dress seeks to examine how technology can be integrated into a garment as an expressive body instrument to augment contemporary sonic performance

Context-Aware Mobile Games Using Android, Arduino and HTML5

Mobiilitehnoloogiate areng ja nutitelefonide kiire levik loob uusi võimalusi mobiilirakenduste arendamiseks ning palju rõhku pannakse mängudele. Loodavad mängud muutuvad aga keerulisemaks, sest üha enam kasutatakse keskkonnast tulevat informatsiooni, et kohandada mängu vastavalt kasutaja asukohale ja kontekstile. Mänge, mis seovad kasutaja füüsile asukoha ja oleku virtuaalmaailmaga keskkonnateadlikuks mänguks. Üha enam luuakse rakendusi veebiplatvormile, mitte kindlatele operatsioonisüsteemidele, sest veebitehnoloogiad võimaldavad luua võrdväärse funktsionaalsuse ning kasutajamugavusega rakendusi. Veebipõhiste rakenduste suurimaks eeliseks on see, et nende tööks on vaja ainult veebibrauserit ning igale operastioonisüteemile ei pea kirjutama eraldi programmi. Antud tehnoloogiad on jõudnud ka mobiilimaailma, kus iga operatsioonisõsteemi jaoks peab arendama eraldiseisva rakenduse, sest iga tootja süsteem on teistest erinev. Antud bakalaurusetöö eesmärgiks on luua veebitehnoloogiaid kasutades mobiilimäng, mille juhtimiseks kasutatakse nutitelefonis olevaid sensoreid ning lisaks Arduino mikrokontrolleritest tulevat väliskeskoona informatsiooni. Antud bakalaurusetöö suurimaks panuseks on PhoneGap raamistikule loodud moodul, mis võimaldab pärida Arduino mikrokontrollerist saadetud sensorite informatsiooni kliendipoolsest koodist, mis tavaliste lahendustega pole võimalik.Latest technological achievements in mobile and open-source electronics platforms made it possible to develop pervasive applications that use environmental information to enhance software usability aspects in real-time, like in the case of context-aware mobile games. However, the development of this kind of pervasive applications is tied to speci c aspects owned by each mobile platform (e.g. programming language, SDK and tools, etc.). Moreover, a considerable effort and knowledge in low-level programming techniques is required for porting the applications between platforms, and thus in general most of the solutions are targeted at particular platform. In order to investigate the possibility of creating portable pervasive applications that combine sensor information from the multiple micromechanical artefacts embedded within the smartphones, we used contextual sensor data provided by Arduino Microcontroller. The current thesis proposed extending the existing implementation of PhoneGap to create hybrid mobile applications based on HTML5 that are easy to port, maintain and reuse

Integrating Usability Models into Pervasive Application Development

This thesis describes novel processes in two important areas of human-computer interaction (HCI) and demonstrates ways to combine these in appropriate ways. First, prototyping plays an essential role in the development of complex applications. This is especially true if a user-centred design process is followed. We describe and compare a set of existing toolkits and frameworks that support the development of prototypes in the area of pervasive computing. Based on these observations, we introduce the EIToolkit that allows the quick generation of mobile and pervasive applications, and approaches many issues found in previous works. Its application and use is demonstrated in several projects that base on the architecture and an implementation of the toolkit. Second, we present novel results and extensions in user modelling, specifically for predicting time to completion of tasks. We extended established concepts such as the Keystroke-Level Model to novel types of interaction with mobile devices, e.g. using optical markers and gestures. The design, creation, as well as a validation of this model are presented in some detail in order to show its use and usefulness for making usability predictions. The third part is concerned with the combination of both concepts, i.e. how to integrate user models into the design process of pervasive applications. We first examine current ways of developing and show generic approaches to this problem. This leads to a concrete implementation of such a solution. An innovative integrated development environment is provided that allows for quickly developing mobile applications, supports the automatic generation of user models, and helps in applying these models early in the design process. This can considerably ease the process of model creation and can replace some types of costly user studies.Diese Dissertation beschreibt neuartige Verfahren in zwei wichtigen Bereichen der Mensch-Maschine-Kommunikation und erläutert Wege, diese geeignet zu verknüpfen. Zum einen spielt die Entwicklung von Prototypen insbesondere bei der Verwendung von benutzerzentrierten Entwicklungsverfahren eine besondere Rolle. Es werden daher auf der einen Seite eine ganze Reihe vorhandener Arbeiten vorgestellt und verglichen, die die Entwicklung prototypischer Anwendungen speziell im Bereich des Pervasive Computing unterstützen. Ein eigener Satz an Werkzeugen und Komponenten wird präsentiert, der viele der herausgearbeiteten Nachteile und Probleme solcher existierender Projekte aufgreift und entsprechende Lösungen anbietet. Mehrere Beispiele und eigene Arbeiten werden beschrieben, die auf dieser Architektur basieren und entwickelt wurden. Auf der anderen Seite werden neue Forschungsergebnisse präsentiert, die Erweiterungen von Methoden in der Benutzermodellierung speziell im Bereich der Abschätzung von Interaktionszeiten beinhalten. Mit diesen in der Dissertation entwickelten Erweiterungen können etablierte Konzepte wie das Keystroke-Level Model auf aktuelle und neuartige Interaktionsmöglichkeiten mit mobilen Geräten angewandt werden. Der Entwurf, das Erstellen sowie eine Validierung der Ergebnisse dieser Erweiterungen werden detailliert dargestellt. Ein dritter Teil beschäftigt sich mit Möglichkeiten die beiden beschriebenen Konzepte, zum einen Prototypenentwicklung im Pervasive Computing und zum anderen Benutzermodellierung, geeignet zu kombinieren. Vorhandene Ansätze werden untersucht und generische Integrationsmöglichkeiten beschrieben. Dies führt zu konkreten Implementierungen solcher Lösungen zur Integration in vorhandene Umgebungen, als auch in Form einer eigenen Applikation spezialisiert auf die Entwicklung von Programmen für mobile Geräte. Sie erlaubt das schnelle Erstellen von Prototypen, unterstützt das automatische Erstellen spezialisierter Benutzermodelle und ermöglicht den Einsatz dieser Modelle früh im Entwicklungsprozess. Dies erleichtert die Anwendung solcher Modelle und kann Aufwand und Kosten für entsprechende Benutzerstudien einsparen

Aesthetic potential of human-computer interaction in performing arts

Human-computer interaction (HCI) is a multidisciplinary area that studies the communication between users and computers. In this thesis, we want to examine if and how HCI when incorporated into staged performances can generate new possibilities for artistic expression on stage. We define and study four areas of technology-enhanced performance that were strongly influenced by HCI techniques: multimedia expression, body representation, body augmentation and interactive environments. We trace relevant artistic practices that contributed to the exploration of these topics and then present new forms of creative expression that emerged after the incorporation of HCI techniques. We present and discuss novel practices like: performer and the media as one responsive entity, real-time control of virtual characters, on-body projections, body augmentation through humanmachine systems and interactive stage design. The thesis concludes by showing some concrete examples of these novel practices implemented in performance pieces. We present and discuss technologyaugmented dance pieces developed during this master’s degree. We also present a software tool for aesthetic visualisation of movement data and discuss its application in video creation, staged performances and interactive installations