Freeform User Interfaces for Graphical Computing

報告番号: 甲15222 ; 学位授与年月日: 2000-03-29 ; 学位の種別: 課程博士 ; 学位の種類: 博士(工学) ; 学位記番号: 博工第4717号 ; 研究科・専攻: 工学系研究科情報工学専

Designing speculative artifacts

In recent decades, Mark Weiser's vision of ubiquitous computing has become today's reality through embedded electronics, the rise of machine learning, and the proliferation of wireless Internet access. This development brings not only opportunities but also new challenges for the Human-Computer Interaction (HCI) community. Technology, for example, is (1) entering novel application scenarios and pristine interaction environments, (2) leaving the screen and conquering the physical world, and (3) changing the previous tool-like nature of computers to social, self-learning, and pro-active entities. As a result, there are no already established interaction paradigms, metaphors, and design strategies for designing such systems. Thus, HCI faces questions about the interaction design in the context of embedded systems and novel materials, the conceptualization of intelligent systems in everyday environments, and, subsequently, the consequences on human-technology relations. To approach such questions, a standard tool in HCI research is the human-centered design process, which creates knowledge about user needs and considers user perspectives to inform design decisions. Observations and interviews are used to understand the context, workflow, or tools, before developing ideas and concepts for technological improvements or solutions. This process has proven to be effective when dealing with matters familiar to users, such as their workplaces, leading to improved workflows and experiences. However, it remains open how design can be grounded if future technologies result in unfamiliar situations. When people can no longer contribute with their domain knowledge, what are novel interaction concepts, paradigms, and designs based on? To tackle these problems, I present and discuss a programmatic design approach to generate original design ideas and concepts. This approach builds on Speculative and Critical Design practices within the HCI context. The main idea central to this work is to create designs using real-world patterns to inform HCI since these patterns still comply with users' prior knowledge, experiences, and perception of fundamental social or natural principles. Such Real-World Patterns (RWP) can be familiar metaphors, morphologies, or mental models - e.g., understanding causality in the physical world or knowing the basic working principles of musical instruments. These patterns are chosen and then transferred into designs to meet or contradict users' expectations of the technology in order to create confrontational situations in which new perspectives are opened up. Based on the confrontation with the speculative artifacts, implications and requirements are deduced, which in reverse can be applied to future technologies. The projects presented apply this approach in various HCI research domains, including human-robot interaction, new interfaces for musical expression, and deformable and flexible interfaces. In this dissertation, I reflect on the approach using three questions: (Q1) How does the use of RWPs complement the Research through Design practice?; (Q2) How can RWPs be instrumentalized in the design of HCI systems?; and (Q3) How does the use of RWPs in design affect the relation between humans and technology? This thesis contributes (1) an overview of the approach as well as three perspectives which are instrumental in understanding and applying RWPs in the design of HCI, (2) eight speculative artifacts, which exemplify the approach, (3) theories and concepts inspired by the used RWPs, and (4) empirical knowledge deduced from the associated studies and surveys.In den letzten Jahrzehnten ist Mark Weisers Vision des Ubiquitous Computing durch eingebettete Systeme, den Aufstieg künstlicher Intelligenz und die Verbreitung des drahtlosen Internetzugangs zur heutigen Lebensrealität geworden. Diese Entwicklung bringt nicht nur Chancen, sondern auch neue Herausforderungen für die Human Computer Interaction (HCI) Community mit sich. Technologien dringen bspw. in neuartige Anwendungsszenarien und noch unberührte Interaktionsumgebungen ein, verlassen den Bildschirm und erobern die physische Welt und verändern so den bisherigen werkzeugartigen Charakter von Computern zu sozialen, selbstlernenden und proaktiven Entitäten. Infolgedessen gibt es keine bereits etablierten Interaktionsparadigmen, Metaphern und Designstrategien, auf die für die Gestaltung solcher Systeme zurückgegriffen werden könnte. Für die HCI stellen sich daher Fragen zur Interaktionsgestaltung im Kontext eingebetteter Systeme und neuartiger Materialien, zur Konzeptualisierung intelligenter Maschinen in Alltagsumgebungen und folglich zu den Auswirkungen auf die Beziehung, in der Mensch und Technologie zueinander stehen. Um sich solchen Fragen zu nähern, ist in der HCI der Human-Centered Design Prozess eine verbreitete Vorgehensweise, die Erkenntnisse über die Bedürfnisse der Benutzer:innen schafft und deren Perspektive bei Designentscheidungen berücksichtigt. Beobachtungen und Interviews werden genutzt, um den Kontext, die Arbeitsabläufe oder Werkzeuge zu verstehen, bevor Ideen und Konzepte für technische Lösungen entwickelt werden. Dieses Vorgehen hat sich als effektiv erwiesen und führt zu Verbesserungen von Technologien, solange die betreffenden Situationen den Nutzer:innen vertraut sind. Offen bleibt jedoch, worauf Design-Konzepte bauen können, wenn zukünftige Technologien zu ungewohnten Situationen führen. Worauf basieren neuartige Interaktionskonzepte, Paradigmen und Entwürfe, wenn Menschen nicht mehr mit ihrem Fachwissen und Erfahrungen beitragen können? Um diese Problematik anzugehen, präsentiere und diskutiere ich in dieser Dissertation einen programmatischen Ansatz, um neuartige Designideen und -konzepte zu entwickeln. Dieser baut auf den Praktiken des spekulativen und kritischen Designs im HCI-Kontext auf. Im Mittelpunkt steht die Entwicklung von Designs unter Verwendung von Mustern aus der realen Welt, da diese das Vorwissen der Nutzer:innen sowie soziale und natürliche Prinzipien einbeziehen. Bei solchen Real-World Patterns (RWP) kann es sich um vertraute Metaphern, Morphologien oder mentale Modelle handeln, bspw. um das Verständnis von Kausalität in der physischen Welt oder um die Kenntnis der grundlegenden Funktionsprinzipien von Musikinstrumenten. Potentielle Muster werden ausgewählt und derart in Entwürfe übertragen, dass die Erwartungen der Nutzer:innen an die Technologie erfüllt oder ihnen widersprochen wird. So werden konfrontative Situationen geschaffen, in denen sich neue Perspektiven eröffnen. Aus der Konfrontation mit den spekulativen Artefakten werden Implikationen und Anforderungen abgeleitet, die sich im Umkehrschluss auf zukünftige Technologien anwenden lassen. Die vorgestellten Projekte wenden diesen Ansatz in verschiedenen HCI-Forschungsbereichen an, unter anderem in der Mensch-Roboter-Interaktion, im Kontext neuer Schnittstellen für die musikalische Interaktion, sowie für deformierbare und flexible Schnittstellen. In dieser Dissertation reflektiere ich den Ansatz anhand dreier Fragen: (Q1) Wie ergänzt der Einsatz von RWPs die Research through Design Praxis?; (Q2) Wie können RWPs für das Design von HCI-Systemen instrumentalisiert werden?; und (Q3) Wie beeinflusst der Einsatz von RWPs die Beziehung zwischen Mensch und Technologie? Der Beitrag dieser Arbeit liegt in (1) einem Überblick über den Ansatz sowie in den drei Perspektiven, die für das Verständnis und die Anwendung von RWPs in der Gestaltung von HCI maßgeblich sind, (2) den acht spekulativen Artefakten, die den Ansatz beispielhaft darstellen, (3) den Theorien und Konzepten, die von den verwendeten RWPs inspiriert sind, und (4) den empirischen Erkenntnissen, die aus den zugehörigen Studien und Umfragen abgeleitet wurden

People Watcher: an app to record and analyzing spatial behavior of ubiquitous interaction technologies

In this paper we argue that interfaces embedded in the world, one of the core objectives of ubiquitous computing, require interaction designers and researchers to have a stronger understanding of the environment as an aspect of the interaction process. We suggest that the interaction community needs new tools to accurately record and, as importantly, analyze interaction in space. We present one solution: People Watcher, a freely downloadable, iPad Application, specifically designed to address the ‘usability in space’ issues. The paper reports a case study of the software’s use. We go on to encourage researchers to adopt this tool as part of the wider process of understanding the effect of the spatial context in interaction design

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Interactive Visual Histories for Vector Graphics

Presentation and graphics software enables users to experiment with variations of illustrations. They can revisit recent editing operations using the ubiquitous undo command, but they are limited to sequential exploration. We propose a new interaction metaphor and visualization for operation history. While editing, a user can access a history mode in which actions are denoted by graphical depictions appearing on top of the document. Our work is inspired by the visual language of film storyboards and assembly instructions. Our storyboard provides an interactive visual history, summarizing the editing of a document or a selected object. Each view is composed of action depictions representing the userâ s editing actions and enables the user to consider the operation history in context rather than in a disconnected list view. This metaphor provides instant access to any past action and we demonstrate that this is an intuitive interface to a selective undo mechanism

Using voice to tag digital photographs on the spot

Tagging of media, particularly digital photographs, has become a very popular and efficient means of organizing material on the internet and on personal computers. Tagging, though, is normally accomplished long after the images have been captured, possibly at the expense of in-the-moment information. Although some digital cameras have begun to automatically populate the various fields of a photograph\u27s metadata, these generic labels often lack in the descriptiveness presented through user-observed annotations and therefore stress the necessity of a user-driven input method. However, most mobile annotation applications demand a great number of keystrokes in order for users to tag photographs and thereby focus the user\u27s attention inward. Specifically, the problem is that these applications require users to take their eyes off the environment while typing in tags. We hypothesize that we can shift the user\u27s focus away from the mobile device and back to their environment by creating a mobile annotation application which accepts voice commands. In other words, our major hypothesis is that a convenient way of tagging digital photographs is by using voice commands