SensaBubble: a chrono-sensory mid-air display of sight and smell

We present SensaBubble, a chrono-sensory mid-air display system that generates scented bubbles to deliver information to the user via a number of sensory modalities. The system reliably produces single bubbles of specific sizes along a directed path. Each bubble produced by SensaBubble is filled with fog containing a scent relevant to the notification. The chrono-sensory aspect of SensaBubble means that information is presented both temporally and multimodally. Temporal information is enabled through two forms of persistence: firstly, a visual display projected onto the bubble which only endures until it bursts; secondly, a scent released upon the bursting of the bubble slowly disperses and leaves a longer-lasting perceptible trace of the event. We report details of SensaBubble’s design and implementation, as well as results of technical and user evaluations. We then discuss and demonstrate how SensaBubble can be adapted for use in a wide range of application contexts – from an ambient peripheral display for persistent alerts, to an engaging display for gaming or education

Broken Display = Broken Interface? The Impact of Display Damage on Smartphone Interaction

This paper is the first to assess the impact of touchscreen damage on smartphone interaction. We gathered a dataset consisting of 95 closeup images of damaged smartphones and extensive information about a device’s usage history, damage severity, and impact on use. 88 % of our participants continued to use their damaged smartphone for at least three months; 32 % plan to use it for another year or more, mainly due to high repair and replacement costs. From the dataset, we identified three categories of damaged smartphone displays. Reading and text input were most affected. Further interviews (n=11) revealed that users adapt to damage with diverse coping strategies, closely tailored to specific interaction issues. In total, we identified 23 different strategies. Based on our results, we proposed guidelines for interaction design in order to provide a positive user experience when display damage occurs. Author Keywords Smartphone; mobile interaction; broken display; displa

Physikit: Data Engagement Through Physical Ambient Visualizations in the Home

Internet of things (IoT) devices and sensor kits have the potential to democratize the access, use, and appropriation of data. Despite the increased availability of low cost sensors, most of the produced data is "black box" in nature: users often do not know how to access or interpret data. We propose a "human-data design" approach in which end-users are given tools to create, share, and use data through tangible and physical visualizations. This paper introduces Physikit, a system designed to allow users to explore and engage with environmental data through physical ambient visualizations. We report on the design and implementation of Physikit, and present a two-week field study which showed that participants got an increased sense of the meaning of data, embellished and appropriated the basic visualizations to make them blend into their homes, and used the visualizations as a probe for community engagement and social behavior

Econundrum:Visualizing the Climate Impact of Dietary Choice through a Shared Data Sculpture

While there is a strong relationship between climate change and human food consumption, it is challenging to understand the implications and impact from an individual perspective. The lack of a shared frame of reference, that allows people to compare their impact to others, limits awareness on this complex topic. To support group reflections and social comparison of the impact of people’s food consumption on climate change, we designed Econundrum, a shared physical data sculpture that visualizes carbon emissions resulting from dietary choices of a small community. Our three-week field study demonstrates how Econundrum helped people (i) understand the climate impact of various food types, (ii) reflect on the environmental impact of their food choices; and (iii) discuss the relation between climate impact and food consumption with others. Our study shows how a shared physical data sculpture mediates a complex topic to a community by facilitating the social dynamics in context

Design for rituals of letting go:an embodiment perspective on disposal practices informed by grief therapy

People increasingly live their lives online, accruing large collections of digital possessions, which symbolically represent important relationships, events, and activities. Most HCI research on bereavement focuses on retaining these significant digital possessions to honor the departed. However recent work suggests that significant digital possessions may complicate moving on; they function as both comforting and painful reminders but currently provide inflexible methods for disposal. Little work has investigated the disposal of digital objects as a means of letting go. To better understand this we interviewed 10 psychotherapists who employ rituals of letting go to help patients overcome loss in situations such as a divorce, a breakup, or a stillbirth. Patients disposed of either natural artefacts or symbolic personal possessions through actions such as burning, burying, or placing in a body of water. Therapists noted people increasingly have digital possessions, and that the act of deletion does not offer the same cathartic sense of release as disposal of material artefacts. Based on analysis of this grief therapy, we propose a new conceptual framework for rituals of letting go that highlights temporality, visibility, and force. It provides a vocabulary to talk about disposal. We then offer design implications connecting the rituals of letting go to the disposal of digital things. Based on our interviews and analytic framework, we propose novel technologies that better connect the embodied nature of letting go rituals to the process of digital disposal

Literature Reviews in HCI: A Review of Reviews

This paper analyses Human-Computer Interaction (HCI) literature reviews to provide a clear conceptual basis for authors, reviewers, and readers. HCI is multidisciplinary and various types of literature reviews exist, from systematic to critical reviews in the style of essays. Yet, there is insufficient consensus of what to expect of literature reviews in HCI. Thus, a shared understanding of literature reviews and clear terminology is needed to plan, evaluate, and use literature reviews, and to further improve review methodology. We analysed 189 literature reviews published at all SIGCHI conferences and ACM Transactions on Computer-Human Interaction (TOCHI) up until August 2022. We report on the main dimensions of variation: (i) contribution types and topics; and (ii) structure and methodologies applied. We identify gaps and trends to inform future meta work in HCI and provide a starting point on how to move towards a more comprehensive terminology system of literature reviews in HCI

muRedder: A Song Ticket Shredding Speaker for Ephemeral Music Contents

Department of Creative Design EngineeringThe digital music streaming service provide an experience to listen to music anytime, anywhere. It also possible to listen to music repeatedly and infinitely. At the same time, tangible media such as LP and CD are constantly being used, and one-time context-based music such as concert, live music and festival music are widely loved. Therefore, in this study, I proposed physical ephemeral digital music and explored the value of the experience of listening to music and consuming music through it. For this, I developed muRedder, a music player that plays song using ephemeral paper song ticket. muRedder is a music player that plays the specific song when the user inserts tangible paper song ticket, and at the same time that shreds the song ticket so that the user could not listen it again. Through the results of field study that recruited 10 participants for 7 days, muRedder helped the user immerse in the music and create a new context and situation to listen to music. In addition, the users selected the song more prudently by using tangible ephemeral song tickets that contained digital music. The contributions of this paper are as follows. First, I proposed a design method that leads to attentive interaction by providing the materialization and ephemeral functional limitation of digital product through the related literature research data. Second, I developed muRedder as an ephemeral music player through the proposed design method, and suggested the issues to consider when designing the research product. Finally, through the field study, the value and the potential of muRedder were proved by ephemeral physical digital content to give user a new experience in music consumption process.clos

Designing speculative artifacts

In recent decades, Mark Weiser's vision of ubiquitous computing has become today's reality through embedded electronics, the rise of machine learning, and the proliferation of wireless Internet access. This development brings not only opportunities but also new challenges for the Human-Computer Interaction (HCI) community. Technology, for example, is (1) entering novel application scenarios and pristine interaction environments, (2) leaving the screen and conquering the physical world, and (3) changing the previous tool-like nature of computers to social, self-learning, and pro-active entities. As a result, there are no already established interaction paradigms, metaphors, and design strategies for designing such systems. Thus, HCI faces questions about the interaction design in the context of embedded systems and novel materials, the conceptualization of intelligent systems in everyday environments, and, subsequently, the consequences on human-technology relations. To approach such questions, a standard tool in HCI research is the human-centered design process, which creates knowledge about user needs and considers user perspectives to inform design decisions. Observations and interviews are used to understand the context, workflow, or tools, before developing ideas and concepts for technological improvements or solutions. This process has proven to be effective when dealing with matters familiar to users, such as their workplaces, leading to improved workflows and experiences. However, it remains open how design can be grounded if future technologies result in unfamiliar situations. When people can no longer contribute with their domain knowledge, what are novel interaction concepts, paradigms, and designs based on? To tackle these problems, I present and discuss a programmatic design approach to generate original design ideas and concepts. This approach builds on Speculative and Critical Design practices within the HCI context. The main idea central to this work is to create designs using real-world patterns to inform HCI since these patterns still comply with users' prior knowledge, experiences, and perception of fundamental social or natural principles. Such Real-World Patterns (RWP) can be familiar metaphors, morphologies, or mental models - e.g., understanding causality in the physical world or knowing the basic working principles of musical instruments. These patterns are chosen and then transferred into designs to meet or contradict users' expectations of the technology in order to create confrontational situations in which new perspectives are opened up. Based on the confrontation with the speculative artifacts, implications and requirements are deduced, which in reverse can be applied to future technologies. The projects presented apply this approach in various HCI research domains, including human-robot interaction, new interfaces for musical expression, and deformable and flexible interfaces. In this dissertation, I reflect on the approach using three questions: (Q1) How does the use of RWPs complement the Research through Design practice?; (Q2) How can RWPs be instrumentalized in the design of HCI systems?; and (Q3) How does the use of RWPs in design affect the relation between humans and technology? This thesis contributes (1) an overview of the approach as well as three perspectives which are instrumental in understanding and applying RWPs in the design of HCI, (2) eight speculative artifacts, which exemplify the approach, (3) theories and concepts inspired by the used RWPs, and (4) empirical knowledge deduced from the associated studies and surveys.In den letzten Jahrzehnten ist Mark Weisers Vision des Ubiquitous Computing durch eingebettete Systeme, den Aufstieg künstlicher Intelligenz und die Verbreitung des drahtlosen Internetzugangs zur heutigen Lebensrealität geworden. Diese Entwicklung bringt nicht nur Chancen, sondern auch neue Herausforderungen für die Human Computer Interaction (HCI) Community mit sich. Technologien dringen bspw. in neuartige Anwendungsszenarien und noch unberührte Interaktionsumgebungen ein, verlassen den Bildschirm und erobern die physische Welt und verändern so den bisherigen werkzeugartigen Charakter von Computern zu sozialen, selbstlernenden und proaktiven Entitäten. Infolgedessen gibt es keine bereits etablierten Interaktionsparadigmen, Metaphern und Designstrategien, auf die für die Gestaltung solcher Systeme zurückgegriffen werden könnte. Für die HCI stellen sich daher Fragen zur Interaktionsgestaltung im Kontext eingebetteter Systeme und neuartiger Materialien, zur Konzeptualisierung intelligenter Maschinen in Alltagsumgebungen und folglich zu den Auswirkungen auf die Beziehung, in der Mensch und Technologie zueinander stehen. Um sich solchen Fragen zu nähern, ist in der HCI der Human-Centered Design Prozess eine verbreitete Vorgehensweise, die Erkenntnisse über die Bedürfnisse der Benutzer:innen schafft und deren Perspektive bei Designentscheidungen berücksichtigt. Beobachtungen und Interviews werden genutzt, um den Kontext, die Arbeitsabläufe oder Werkzeuge zu verstehen, bevor Ideen und Konzepte für technische Lösungen entwickelt werden. Dieses Vorgehen hat sich als effektiv erwiesen und führt zu Verbesserungen von Technologien, solange die betreffenden Situationen den Nutzer:innen vertraut sind. Offen bleibt jedoch, worauf Design-Konzepte bauen können, wenn zukünftige Technologien zu ungewohnten Situationen führen. Worauf basieren neuartige Interaktionskonzepte, Paradigmen und Entwürfe, wenn Menschen nicht mehr mit ihrem Fachwissen und Erfahrungen beitragen können? Um diese Problematik anzugehen, präsentiere und diskutiere ich in dieser Dissertation einen programmatischen Ansatz, um neuartige Designideen und -konzepte zu entwickeln. Dieser baut auf den Praktiken des spekulativen und kritischen Designs im HCI-Kontext auf. Im Mittelpunkt steht die Entwicklung von Designs unter Verwendung von Mustern aus der realen Welt, da diese das Vorwissen der Nutzer:innen sowie soziale und natürliche Prinzipien einbeziehen. Bei solchen Real-World Patterns (RWP) kann es sich um vertraute Metaphern, Morphologien oder mentale Modelle handeln, bspw. um das Verständnis von Kausalität in der physischen Welt oder um die Kenntnis der grundlegenden Funktionsprinzipien von Musikinstrumenten. Potentielle Muster werden ausgewählt und derart in Entwürfe übertragen, dass die Erwartungen der Nutzer:innen an die Technologie erfüllt oder ihnen widersprochen wird. So werden konfrontative Situationen geschaffen, in denen sich neue Perspektiven eröffnen. Aus der Konfrontation mit den spekulativen Artefakten werden Implikationen und Anforderungen abgeleitet, die sich im Umkehrschluss auf zukünftige Technologien anwenden lassen. Die vorgestellten Projekte wenden diesen Ansatz in verschiedenen HCI-Forschungsbereichen an, unter anderem in der Mensch-Roboter-Interaktion, im Kontext neuer Schnittstellen für die musikalische Interaktion, sowie für deformierbare und flexible Schnittstellen. In dieser Dissertation reflektiere ich den Ansatz anhand dreier Fragen: (Q1) Wie ergänzt der Einsatz von RWPs die Research through Design Praxis?; (Q2) Wie können RWPs für das Design von HCI-Systemen instrumentalisiert werden?; und (Q3) Wie beeinflusst der Einsatz von RWPs die Beziehung zwischen Mensch und Technologie? Der Beitrag dieser Arbeit liegt in (1) einem Überblick über den Ansatz sowie in den drei Perspektiven, die für das Verständnis und die Anwendung von RWPs in der Gestaltung von HCI maßgeblich sind, (2) den acht spekulativen Artefakten, die den Ansatz beispielhaft darstellen, (3) den Theorien und Konzepten, die von den verwendeten RWPs inspiriert sind, und (4) den empirischen Erkenntnissen, die aus den zugehörigen Studien und Umfragen abgeleitet wurden

Tracking in the wild: exploring the everyday use of physical activity trackers

As the rates of chronical diseases, such as obesity, cardiovascular disease and diabetes continue to increase, the development of tools that support people in achieving healthier habits is becoming ever more important. Personal tracking systems, such as activity trackers, have emerged as a promising class of tools to support people in managing their everyday health. However, for this promise to be fulfilled, these systems need to be well designed, not only in terms of how they implement specific behavior change techniques, but also in how they integrate into people’s daily lives and address their daily needs. My dissertations provides evidence that accounting for people’s daily practices and needs can help to design activity tracking systems that help people get more value from their tracking practices. To understand how people derive value from their activity tracking practices, I have conducted two inquiries into people’s daily uses of activity tracking systems. In a fist attempt, I led a 10-month study of the adoption of Habito, our own activity tracking mobile app. Habito logged not only users’ physical activity, but also their interactions with the app. This data was used to acquire an estimate of the adoption rate of Habito, and understanding of how adoption is affected by users’ ‘readiness’, i.e., their attitude towards behavior change. In a follow-up study, I turned to the use of video methods and direct, in-situ observations of users’ interactions to understand what motivates people to engage with these tools in their everyday life, and how the surrounding environment shapes their use. These studies revealed some of the complexities of tracking, while extending some of the underlying ideas of behavior change. Among key results: (1) people’s use of activity trackers was found to be predominantly impulsive, where they simultaneously reflect, learn and change their behaviors as they collect data; (2) people’s use of trackers is deeply entangled with their daily routines and practices, and; (3) people use of trackers often is not in line with the traditional vision of these tools as mediators of change – trackers are also commonly used to simply learn about behaviors and engage in moments of self-discovery. Examining how to design activity tracking interfaces that best support people’s different needs , my dissertation further describes an inquiry into the design space of behavioral feedback interfaces. Through a iterative process of synthesis and analysis of research on activity tracking, I devise six design qualities for creating feedback that supports people in their interactions with physical activity data. Through the development and field deployment of four concepts in a field study, I show the potential of these displays for highlighting opportunities for action and learning.À medida que a prevalência de doenças crónicas como a obesidade, doenças cardiovasculares e diabetes continua a aumentar, o desenvolvimento de ferramentas que suportam pessoas a atingir mudanças de comportamento tem-se tornado essencial. Ferramentas de monitorização de comportamentos, tais como monitores de atividade física, têm surgido com a promessa de encorajar um dia a dia mais saudável. Contudo, para que essa promessa seja cumprida, torna-se essencial que estas ferramentas sejam bem concebidas, não só na forma como implementam determinadas estratégias de mudança de comportamento, mas também na forma como são integradas no dia-a-dia das pessoas. A minha dissertação demonstra a importância de considerar as necessidades e práticas diárias dos utilizadores destas ferramentas, de forma a ajudá-las a tirar melhor proveito da sua monitorização de atividade física. De modo a entender como é que os utilizadores destas ferramentas derivam valor das suas práticas de monitorização, a minha dissertação começa por explorar as práticas diárias associadas ao uso de monitores de atividade física. A minha dissertação contribui com duas investigações ao uso diário destas ferramentas. Primeiro, é apresentada uma investigação da adoção de Habito, uma aplicação para monitorização de atividade física. Habito não só registou as instâncias de atividade física dos seus utilizadores, mas também as suas interações com a própria aplicação. Estes dados foram utilizados para adquirir uma taxa de adopção de Habito e entender como é que essa adopção é afetada pela “prontidão” dos utilizadores, i.e., a sua atitude em relação à mudança de comportamento. Num segundo estudo, recorrendo a métodos de vídeo e observações diretas e in-situ da utilização de monitores de atividade física, explorei as motivações associadas ao uso diário destas ferramentas. Estes estudos expandiram algumas das ideias subjacentes ao uso das ferramentas para mudanças de comportamento. Entre resultados principais: (1) o uso de monitores de atividade física é predominantemente impulsivo, onde pessoas refletem, aprendem e alteram os seus comportamentos à medida que recolhem dados sobe estes mesmos comportamentos; (2) o uso de monitores de atividade física está profundamente interligado com as rotinas e práticas dos seus utilizadores, e; (3) o uso de monitores de atividade física nem sempre está ligado a mudanças de comportamento – estas ferramentas também são utilizadas para divertimento e aprendizagem. A minha dissertação contribui ainda com uma exploração do design de interfaces para a monitorização de atividade física. Através de um processo iterativo de síntese e análise de literatura, seis qualidades para a criação de interfaces são derivadas. Através de um estudo de campo, a minha dissertação demonstro o potencial dessas interfaces para ajudar pessoas a aprender e gerir a sua saúde diária

Literature Reviews in HCI: A Review of Reviews

This paper analyses Human-Computer Interaction (HCI) literature reviews to provide a clear conceptual basis for authors, reviewers, and readers. HCI is multidisciplinary and various types of literature reviews exist, from systematic to critical reviews in the style of essays. Yet, there is insufficient consensus of what to expect of literature reviews in HCI. Thus, a shared understanding of literature reviews and clear terminology is needed to plan, evaluate, and use literature reviews, and to further improve review methodology. We analysed 189 literature reviews published at all SIGCHI conferences and ACM Transactions on Computer-Human Interaction (TOCHI) up until August 2022. We report on the main dimensions of variation: (i) contribution types and topics; and (ii) structure and methodologies applied. We identify gaps and trends to inform future meta work in HCI and provide a starting point on how to move towards a more comprehensive terminology system of literature reviews in HCI