Understanding receptivity to interruptions in mobile human-computer interaction

Interruptions have a profound impact on our attentional orientation in everyday life. Recent advances in mobile information technology increase the number of potentially disruptive notifications on mobile devices by an increasing availability of services. Understanding the contextual intricacies that make us receptive to these interruptions is paramount to devising technology that supports interruption management. This thesis makes a number of contributions to the methodology of studying mobile experiences in situ, understanding receptivity to interruptions, and designing context-sensitive systems. This thesis presents a series of real-world studies that investigate opportune moments for interruptions in mobile settings. In order to facilitate the study of the multi-faceted ways opportune moments surface from participants' involvement in the world this thesis develops: - a model of the contextual factors that interact to guide receptivity to interruptions, and - an adaptation of the Experience-Sampling Method (ESM) to capture behavioural response to interruptions in situ. In two naturalistic experiments, participants' experiences of being interrupted on a mobile phone are sampled as they go about their everyday lives. In a field study, participants' experiences are observed and recorded as they use a notification-driven mobile application to create photo-stories in a theme park. Experiment 1 explores the effects of content and time of delivery of the interruption. The results show that receptivity to text messages is significantly affected by message content, while scheduling one's own interruption times in advance does not improve receptivity over randomly timed interruptions. Experiment 2 investigates the hypothesis that opportune moments to deliver notifications are located at the endings of episodes of mobile interaction such as texting and calling. This notification strategy is supported by significant effects in behavioural measures of receptivity, while self-reports and interviews reveal complexities in the subjective experience of the interruption. By employing a mixed methods approach of interviews, observations and an analysis of system logs in the field study, it is shown that participants appreciated location-based notifications as prompts to foreground the application during relative 'downtimes' from other activities. However, an unexpected quantity of redundant notifications meant that visitors soon habituated to and eventually ignored them, which suggests careful, sparing use of notifications in interactive experiences. Overall, the studies showed that contextual mediation of the timing of interruptions (e.g. by phone activity in Experiment 2 and opportune places in the field study) is more likely to lead to interruptions at opportune moments than when participants schedule their own interruptions. However, momentary receptivity and responsiveness to an interruption is determined by the complex and situated interactions of local and relational contextual factors. These contextual factors are captured in a model of receptivity that underlies the interruption process. The studies highlight implications for the design of systems that seek to manage interruptions by adapting the timing of interruptions to the user's situation. In particular, applications to manage interruptions in personal communication and pervasive experiences are considered

Understanding receptivity to interruptions in mobile human-computer interaction

Interruptions have a profound impact on our attentional orientation in everyday life. Recent advances in mobile information technology increase the number of potentially disruptive notifications on mobile devices by an increasing availability of services. Understanding the contextual intricacies that make us receptive to these interruptions is paramount to devising technology that supports interruption management. This thesis makes a number of contributions to the methodology of studying mobile experiences in situ, understanding receptivity to interruptions, and designing context-sensitive systems. This thesis presents a series of real-world studies that investigate opportune moments for interruptions in mobile settings. In order to facilitate the study of the multi-faceted ways opportune moments surface from participants' involvement in the world this thesis develops: - a model of the contextual factors that interact to guide receptivity to interruptions, and - an adaptation of the Experience-Sampling Method (ESM) to capture behavioural response to interruptions in situ. In two naturalistic experiments, participants' experiences of being interrupted on a mobile phone are sampled as they go about their everyday lives. In a field study, participants' experiences are observed and recorded as they use a notification-driven mobile application to create photo-stories in a theme park. Experiment 1 explores the effects of content and time of delivery of the interruption. The results show that receptivity to text messages is significantly affected by message content, while scheduling one's own interruption times in advance does not improve receptivity over randomly timed interruptions. Experiment 2 investigates the hypothesis that opportune moments to deliver notifications are located at the endings of episodes of mobile interaction such as texting and calling. This notification strategy is supported by significant effects in behavioural measures of receptivity, while self-reports and interviews reveal complexities in the subjective experience of the interruption. By employing a mixed methods approach of interviews, observations and an analysis of system logs in the field study, it is shown that participants appreciated location-based notifications as prompts to foreground the application during relative 'downtimes' from other activities. However, an unexpected quantity of redundant notifications meant that visitors soon habituated to and eventually ignored them, which suggests careful, sparing use of notifications in interactive experiences. Overall, the studies showed that contextual mediation of the timing of interruptions (e.g. by phone activity in Experiment 2 and opportune places in the field study) is more likely to lead to interruptions at opportune moments than when participants schedule their own interruptions. However, momentary receptivity and responsiveness to an interruption is determined by the complex and situated interactions of local and relational contextual factors. These contextual factors are captured in a model of receptivity that underlies the interruption process. The studies highlight implications for the design of systems that seek to manage interruptions by adapting the timing of interruptions to the user's situation. In particular, applications to manage interruptions in personal communication and pervasive experiences are considered

Decomposing responses to mobile notifications

Notifications from mobile devices frequently prompt us with information, either to merely inform us or to elicit a reaction. This has led to increasing research interest in considering an individual’s interruptibility prior to issuing notifications, in order for them to be positively received. To achieve this, predictive models need to be built from previous response behaviour where the individual’s interruptibility is known. However, there are several degrees of freedom in achieving this, from different definitions in what it means to be interruptible and a notification to be successful, to various methods for collecting data, and building predictive models. The primary focus of this thesis is to improve upon the typical convention used for labelling interruptibility, an area which has had limited direct attention. This includes the proposal of a flexible framework, called the decision-on-information-gain model, which passively observes response behaviour in order to support various interruptibility definitions. In contrast, previous studies have largely surrounded the investigation of influential contextual factors on predicting interruptibility, using a broad labelling convention that relies on notifications being responded to fully and potentially a survey needing to be completed. The approach is supported through two in-the-wild studies of Android notifications, one with 11,000 notifications across 90 users, and another with 32,000,000 across 3000 users. Analysis of these datasets shows that: a) responses to notifications is a decisionmaking process, whereby individuals can be reachable but not receptive to their content, supporting the premise of the approach; b) the approach is implementable on typical Android devices and capable of adapting to different notification designs and user preferences; and c) the different labels produced by the model are predictable using data sources that do not require invasive permissions or persistent background monitoring; however there are notable performance differences between different machine learning strategies for training and evaluation

Pssst...or Boo! Assessing the Predictability of Notification Delivery Preferences.

The focus of my dissertation research is on the examination of notification systems that harness different presentation formats for notification delivery, the preferences that individuals express for these various types of notifications, and how these preferences are affected by contextual information surrounding notification delivery. My research is unique from other work in the literature in two primary ways. First, while the majority of prior work addressing notification delivery, both in terms of format and timing, has focused on the effects of a notification on an individual’s performance on a given task or set of tasks, my focus is the individual’s perception of notifications, and particularly on that individual’s preferences for different notification formats delivered within different contextual scenarios. An interest in this question is motivated by prior studies that have shown that annoyance with computer-human interactions is a primary reason behind user abandonment of interactive software systems. Second, my preliminary findings suggest that different people prefer different types of notifications in different contexts, which motivates a change of focus in the development of such systems toward customizing notifications not only to the features of an individual’s context but also to the individual him- or herself. An additional element of novelty in my work is that my final study was conducted in a purely naturalistic office environment, in which the notifications evaluated were precisely those notifications being delivered to study participants throughout their workday. The primary contribution of this dissertation is twofold: a detailed analysis of the methodology for the design, data collection, and analysis of a study of notification preferences in a naturalistic setting with a great deal of inherent complexity; and a set of results, based on the analysis of preference data acquired in various settings, about how an individual’s contextual environment, and the content of a given notification, can affect that individual’s preferences for notification delivery.Ph.D.Computer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/78809/1/weberjs_1.pd

Foundations of Human-Aware Planning -- A Tale of Three Models

abstract: A critical challenge in the design of AI systems that operate with humans in the loop is to be able to model the intentions and capabilities of the humans, as well as their beliefs and expectations of the AI system itself. This allows the AI system to be "human- aware" -- i.e. the human task model enables it to envisage desired roles of the human in joint action, while the human mental model allows it to anticipate how its own actions are perceived from the point of view of the human. In my research, I explore how these concepts of human-awareness manifest themselves in the scope of planning or sequential decision making with humans in the loop. To this end, I will show (1) how the AI agent can leverage the human task model to generate symbiotic behavior; and (2) how the introduction of the human mental model in the deliberative process of the AI agent allows it to generate explanations for a plan or resort to explicable plans when explanations are not desired. The latter is in addition to traditional notions of human-aware planning which typically use the human task model alone and thus enables a new suite of capabilities of a human-aware AI agent. Finally, I will explore how the AI agent can leverage emerging mixed-reality interfaces to realize effective channels of communication with the human in the loop.Dissertation/ThesisDoctoral Dissertation Computer Science 201

The selective use of gaze in automatic speech recognition

The performance of automatic speech recognition (ASR) degrades significantly in natural environments compared to in laboratory assessments. Being a major source of interference, acoustic noise affects speech intelligibility during the ASR process. There are two main problems caused by the acoustic noise. The first is the speech signal contamination. The second is the speakers' vocal and non-vocal behavioural changes. These phenomena elicit mismatch between the ASR training and recognition conditions, which leads to considerable performance degradation. To improve noise-robustness, exploiting prior knowledge of the acoustic noise in speech enhancement, feature extraction and recognition models are popular approaches. An alternative approach presented in this thesis is to introduce eye gaze as an extra modality. Eye gaze behaviours have roles in interaction and contain information about cognition and visual attention; not all behaviours are relevant to speech. Therefore, gaze behaviours are used selectively to improve ASR performance. This is achieved by inference procedures using noise-dependant models of gaze behaviours and their temporal and semantic relationship with speech. `Selective gaze-contingent ASR' systems are proposed and evaluated on a corpus of eye movement and related speech in different clean, noisy environments. The best performing systems utilise both acoustic and language model adaptation

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools