Workload-aware systems and interfaces for cognitive augmentation

In today's society, our cognition is constantly influenced by information intake, attention switching, and task interruptions. This increases the difficulty of a given task, adding to the existing workload and leading to compromised cognitive performances. The human body expresses the use of cognitive resources through physiological responses when confronted with a plethora of cognitive workload. This temporarily mobilizes additional resources to deal with the workload at the cost of accelerated mental exhaustion. We predict that recent developments in physiological sensing will increasingly create user interfaces that are aware of the user’s cognitive capacities, hence able to intervene when high or low states of cognitive workload are detected. In this thesis, we initially focus on determining opportune moments for cognitive assistance. Subsequently, we investigate suitable feedback modalities in a user-centric design process which are desirable for cognitive assistance. We present design requirements for how cognitive augmentation can be achieved using interfaces that sense cognitive workload. We then investigate different physiological sensing modalities to enable suitable real-time assessments of cognitive workload. We provide empirical evidence that the human brain is sensitive to fluctuations in cognitive resting states, hence making cognitive effort measurable. Firstly, we show that electroencephalography is a reliable modality to assess the mental workload generated during the user interface operation. Secondly, we use eye tracking to evaluate changes in eye movements and pupil dilation to quantify different workload states. The combination of machine learning and physiological sensing resulted in suitable real-time assessments of cognitive workload. The use of physiological sensing enables us to derive when cognitive augmentation is suitable. Based on our inquiries, we present applications that regulate cognitive workload in home and work settings. We deployed an assistive system in a field study to investigate the validity of our derived design requirements. Finding that workload is mitigated, we investigated how cognitive workload can be visualized to the user. We present an implementation of a biofeedback visualization that helps to improve the understanding of brain activity. A final study shows how cognitive workload measurements can be used to predict the efficiency of information intake through reading interfaces. Here, we conclude with use cases and applications which benefit from cognitive augmentation. This thesis investigates how assistive systems can be designed to implicitly sense and utilize cognitive workload for input and output. To do so, we measure cognitive workload in real-time by collecting behavioral and physiological data from users and analyze this data to support users through assistive systems that adapt their interface according to the currently measured workload. Our overall goal is to extend new and existing context-aware applications by the factor cognitive workload. We envision Workload-Aware Systems and Workload-Aware Interfaces as an extension in the context-aware paradigm. To this end, we conducted eight research inquiries during this thesis to investigate how to design and create workload-aware systems. Finally, we present our vision of future workload-aware systems and workload-aware interfaces. Due to the scarce availability of open physiological data sets, reference implementations, and methods, previous context-aware systems were limited in their ability to utilize cognitive workload for user interaction. Together with the collected data sets, we expect this thesis to pave the way for methodical and technical tools that integrate workload-awareness as a factor for context-aware systems.Tagtäglich werden unsere kognitiven Fähigkeiten durch die Verarbeitung von unzähligen Informationen in Anspruch genommen. Dies kann die Schwierigkeit einer Aufgabe durch mehr oder weniger Arbeitslast beeinflussen. Der menschliche Körper drückt die Nutzung kognitiver Ressourcen durch physiologische Reaktionen aus, wenn dieser mit kognitiver Arbeitsbelastung konfrontiert oder überfordert wird. Dadurch werden weitere Ressourcen mobilisiert, um die Arbeitsbelastung vorübergehend zu bewältigen. Wir prognostizieren, dass die derzeitige Entwicklung physiologischer Messverfahren kognitive Leistungsmessungen stets möglich machen wird, um die kognitive Arbeitslast des Nutzers jederzeit zu messen. Diese sind in der Lage, einzugreifen wenn eine zu hohe oder zu niedrige kognitive Belastung erkannt wird. Wir konzentrieren uns zunächst auf die Erkennung passender Momente für kognitive Unterstützung welche sich der gegenwärtigen kognitiven Arbeitslast bewusst sind. Anschließend untersuchen wir in einem nutzerzentrierten Designprozess geeignete Feedbackmechanismen, die zur kognitiven Assistenz beitragen. Wir präsentieren Designanforderungen, welche zeigen wie Schnittstellen eine kognitive Augmentierung durch die Messung kognitiver Arbeitslast erreichen können. Anschließend untersuchen wir verschiedene physiologische Messmodalitäten, welche Bewertungen der kognitiven Arbeitsbelastung in Realzeit ermöglichen. Zunächst validieren wir empirisch, dass das menschliche Gehirn auf kognitive Arbeitslast reagiert. Es zeigt sich, dass die Ableitung der kognitiven Arbeitsbelastung über Elektroenzephalographie eine geeignete Methode ist, um den kognitiven Anspruch neuartiger Assistenzsysteme zu evaluieren. Anschließend verwenden wir Eye-Tracking, um Veränderungen in den Augenbewegungen und dem Durchmesser der Pupille unter verschiedenen Intensitäten kognitiver Arbeitslast zu bewerten. Das Anwenden von maschinellem Lernen führt zu zuverlässigen Echtzeit-Bewertungen kognitiver Arbeitsbelastung. Auf der Grundlage der bisherigen Forschungsarbeiten stellen wir Anwendungen vor, welche die Kognition im häuslichen und beruflichen Umfeld unterstützen. Die physiologischen Messungen stellen fest, wann eine kognitive Augmentierung sich als günstig erweist. In einer Feldstudie setzen wir ein Assistenzsystem ein, um die erhobenen Designanforderungen zur Reduktion kognitiver Arbeitslast zu validieren. Unsere Ergebnisse zeigen, dass die Arbeitsbelastung durch den Einsatz von Assistenzsystemen reduziert wird. Im Anschluss untersuchen wir, wie kognitive Arbeitsbelastung visualisiert werden kann. Wir stellen eine Implementierung einer Biofeedback-Visualisierung vor, die das Nutzerverständnis zum Verlauf und zur Entstehung von kognitiver Arbeitslast unterstützt. Eine abschließende Studie zeigt, wie Messungen kognitiver Arbeitslast zur Vorhersage der aktuellen Leseeffizienz benutzt werden können. Wir schließen hierbei mit einer Reihe von Applikationen ab, welche sich kognitive Arbeitslast als Eingabe zunutze machen. Die vorliegende wissenschaftliche Arbeit befasst sich mit dem Design von Assistenzsystemen, welche die kognitive Arbeitslast der Nutzer implizit erfasst und diese bei der Durchführung alltäglicher Aufgaben unterstützt. Dabei werden physiologische Daten erfasst, um Rückschlüsse in Realzeit auf die derzeitige kognitive Arbeitsbelastung zu erlauben. Anschließend werden diese Daten analysiert, um dem Nutzer strategisch zu assistieren. Das Ziel dieser Arbeit ist die Erweiterung neuartiger und bestehender kontextbewusster Benutzerschnittstellen um den Faktor kognitive Arbeitslast. Daher werden in dieser Arbeit arbeitslastbewusste Systeme und arbeitslastbewusste Benutzerschnittstellen als eine zusätzliche Dimension innerhalb des Paradigmas kontextbewusster Systeme präsentiert. Wir stellen acht Forschungsstudien vor, um die Designanforderungen und die Implementierung von kognitiv arbeitslastbewussten Systemen zu untersuchen. Schließlich stellen wir unsere Vision von zukünftigen kognitiven arbeitslastbewussten Systemen und Benutzerschnittstellen vor. Durch die knappe Verfügbarkeit öffentlich zugänglicher Datensätze, Referenzimplementierungen, und Methoden, waren Kontextbewusste Systeme in der Auswertung kognitiver Arbeitslast bezüglich der Nutzerinteraktion limitiert. Ergänzt durch die in dieser Arbeit gesammelten Datensätze erwarten wir, dass diese Arbeit den Weg für methodische und technische Werkzeuge ebnet, welche kognitive Arbeitslast als Faktor in das Kontextbewusstsein von Computersystemen integriert

Understanding receptivity to interruptions in mobile human-computer interaction

Interruptions have a profound impact on our attentional orientation in everyday life. Recent advances in mobile information technology increase the number of potentially disruptive notifications on mobile devices by an increasing availability of services. Understanding the contextual intricacies that make us receptive to these interruptions is paramount to devising technology that supports interruption management. This thesis makes a number of contributions to the methodology of studying mobile experiences in situ, understanding receptivity to interruptions, and designing context-sensitive systems. This thesis presents a series of real-world studies that investigate opportune moments for interruptions in mobile settings. In order to facilitate the study of the multi-faceted ways opportune moments surface from participants' involvement in the world this thesis develops: - a model of the contextual factors that interact to guide receptivity to interruptions, and - an adaptation of the Experience-Sampling Method (ESM) to capture behavioural response to interruptions in situ. In two naturalistic experiments, participants' experiences of being interrupted on a mobile phone are sampled as they go about their everyday lives. In a field study, participants' experiences are observed and recorded as they use a notification-driven mobile application to create photo-stories in a theme park. Experiment 1 explores the effects of content and time of delivery of the interruption. The results show that receptivity to text messages is significantly affected by message content, while scheduling one's own interruption times in advance does not improve receptivity over randomly timed interruptions. Experiment 2 investigates the hypothesis that opportune moments to deliver notifications are located at the endings of episodes of mobile interaction such as texting and calling. This notification strategy is supported by significant effects in behavioural measures of receptivity, while self-reports and interviews reveal complexities in the subjective experience of the interruption. By employing a mixed methods approach of interviews, observations and an analysis of system logs in the field study, it is shown that participants appreciated location-based notifications as prompts to foreground the application during relative 'downtimes' from other activities. However, an unexpected quantity of redundant notifications meant that visitors soon habituated to and eventually ignored them, which suggests careful, sparing use of notifications in interactive experiences. Overall, the studies showed that contextual mediation of the timing of interruptions (e.g. by phone activity in Experiment 2 and opportune places in the field study) is more likely to lead to interruptions at opportune moments than when participants schedule their own interruptions. However, momentary receptivity and responsiveness to an interruption is determined by the complex and situated interactions of local and relational contextual factors. These contextual factors are captured in a model of receptivity that underlies the interruption process. The studies highlight implications for the design of systems that seek to manage interruptions by adapting the timing of interruptions to the user's situation. In particular, applications to manage interruptions in personal communication and pervasive experiences are considered

Understanding receptivity to interruptions in mobile human-computer interaction

Interruptions have a profound impact on our attentional orientation in everyday life. Recent advances in mobile information technology increase the number of potentially disruptive notifications on mobile devices by an increasing availability of services. Understanding the contextual intricacies that make us receptive to these interruptions is paramount to devising technology that supports interruption management. This thesis makes a number of contributions to the methodology of studying mobile experiences in situ, understanding receptivity to interruptions, and designing context-sensitive systems. This thesis presents a series of real-world studies that investigate opportune moments for interruptions in mobile settings. In order to facilitate the study of the multi-faceted ways opportune moments surface from participants' involvement in the world this thesis develops: - a model of the contextual factors that interact to guide receptivity to interruptions, and - an adaptation of the Experience-Sampling Method (ESM) to capture behavioural response to interruptions in situ. In two naturalistic experiments, participants' experiences of being interrupted on a mobile phone are sampled as they go about their everyday lives. In a field study, participants' experiences are observed and recorded as they use a notification-driven mobile application to create photo-stories in a theme park. Experiment 1 explores the effects of content and time of delivery of the interruption. The results show that receptivity to text messages is significantly affected by message content, while scheduling one's own interruption times in advance does not improve receptivity over randomly timed interruptions. Experiment 2 investigates the hypothesis that opportune moments to deliver notifications are located at the endings of episodes of mobile interaction such as texting and calling. This notification strategy is supported by significant effects in behavioural measures of receptivity, while self-reports and interviews reveal complexities in the subjective experience of the interruption. By employing a mixed methods approach of interviews, observations and an analysis of system logs in the field study, it is shown that participants appreciated location-based notifications as prompts to foreground the application during relative 'downtimes' from other activities. However, an unexpected quantity of redundant notifications meant that visitors soon habituated to and eventually ignored them, which suggests careful, sparing use of notifications in interactive experiences. Overall, the studies showed that contextual mediation of the timing of interruptions (e.g. by phone activity in Experiment 2 and opportune places in the field study) is more likely to lead to interruptions at opportune moments than when participants schedule their own interruptions. However, momentary receptivity and responsiveness to an interruption is determined by the complex and situated interactions of local and relational contextual factors. These contextual factors are captured in a model of receptivity that underlies the interruption process. The studies highlight implications for the design of systems that seek to manage interruptions by adapting the timing of interruptions to the user's situation. In particular, applications to manage interruptions in personal communication and pervasive experiences are considered

Cognition-aware systems to support information intake and learning

Knowledge is created at an ever-increasing pace putting us under constant pressure to consume and acquire new information. Information gain and learning, however, require time and mental resources. While the proliferation of ubiquitous computing devices, such as smartphones, enables us to consume information anytime and anywhere, technologies are often disruptive rather than sensitive to the current user context. While people exhibit different levels of concentration and cognitive capacity throughout the day, applications rarely take these performance variations into account and often overburden their users with information or fail to stimulate. This work investigates how technology can be used to help people effectively deal with information intake and learning tasks through cognitive context-awareness. By harvesting sensor and usage data from mobile devices, we obtain people's levels of attentiveness, receptiveness, and cognitive performance. We subsequently use this cognition-awareness in applications to help users process information more effectively. Through a series of lab studies, online surveys, and field experiments we follow six research questions to investigate how to build cognition-aware systems. Awareness of user's variations in levels of attention, receptiveness, and cognitive performance allows systems to trigger appropriate content suggestions, manage user interruptions, and adapt User Interfaces in real-time to match tasks to the user's cognitive capacities. The tools, insights, and concepts described in this book allow researchers and application designers to build systems with an awareness of momentary user states and general circadian rhythms of alertness and cognitive performance

Proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching (IWEPLET 2013)

"This book contains the proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching (IWEPLET) 2013 which was held on 16.-17.September 2013 in Paphos (Cyprus) in conjunction with the EC-TEL conference. The workshop and hence the proceedings are divided in two parts: on Day 1 the EuroPLOT project and its results are introduced, with papers about the specific case studies and their evaluation. On Day 2, peer-reviewed papers are presented which address specific topics and issues going beyond the EuroPLOT scope. This workshop is one of the deliverables (D 2.6) of the EuroPLOT project, which has been funded from November 2010 – October 2013 by the Education, Audiovisual and Culture Executive Agency (EACEA) of the European Commission through the Lifelong Learning Programme (LLL) by grant #511633. The purpose of this project was to develop and evaluate Persuasive Learning Objects and Technologies (PLOTS), based on ideas of BJ Fogg. The purpose of this workshop is to summarize the findings obtained during this project and disseminate them to an interested audience. Furthermore, it shall foster discussions about the future of persuasive technology and design in the context of learning, education and teaching. The international community working in this area of research is relatively small. Nevertheless, we have received a number of high-quality submissions which went through a peer-review process before being selected for presentation and publication. We hope that the information found in this book is useful to the reader and that more interest in this novel approach of persuasive design for teaching/education/learning is stimulated. We are very grateful to the organisers of EC-TEL 2013 for allowing to host IWEPLET 2013 within their organisational facilities which helped us a lot in preparing this event. I am also very grateful to everyone in the EuroPLOT team for collaborating so effectively in these three years towards creating excellent outputs, and for being such a nice group with a very positive spirit also beyond work. And finally I would like to thank the EACEA for providing the financial resources for the EuroPLOT project and for being very helpful when needed. This funding made it possible to organise the IWEPLET workshop without charging a fee from the participants.

A study of a christian conception of vocation in networked life: in dialogue with Oliver O’Donovan and Robert Adams

In networked life, individuals are encouraged to live with a sense of limitlessness, and yet are provided with less, agency diminished, if not corrupted. Action within networked life seeks control or dominance in the material realm, such short-termism antithetical to freedom. These challenges are clearly presented, albeit from contrasting angles, in the work of the psychoanalyst Sherry Turkle and the sociologist Ronald Burt (contextualised by Marshall McLuhan, Jacques Ellul and George Grant), each converging on control as the aim of networked life. We discover that networked life is, in its corrupted state, a life in the flesh. Networked life, however, can be redeemed. In this dissertation, I argue that a certain Christian conception of vocation is the way through networked life. Through discussion with Robert Adams and Oliver O’Donovan, vocation is defined as ‘An offering from God, involving an ensemble of worldly relations and created goods, through which we are given, in particular, to pursue excellence, realise our agency and serve God.’ I stress Adams’ focus on pursuit of excellence, but in a manner that is reflective of O’Donovan’s restraint. Both interlocutors provide compelling conceptions of vocation, with ample opportunity for critical dialogue (this contextualised by Grisez, Bonhoeffer, Ellul, Stackhouse, Jr., Williams and others), particularly in areas such as faith, love and responsibility. Moreover, Adams proposes a theory of value grounding excellence in resemblance to the Transcendent Good, whereas O’Donovan embeds value in moral order, which seizes individuals. Each sees the measure of value as residing outside of individuals, discussion on value important in a world that otherwise prioritises the securing of short-term material advantage. This dissertation favours an evangelical Christian ethics, engaging constructively with networked life, and making use of its clear benefits, using such benefits for mutual service in the world, and in the ultimate service of God

Smartphones as steady companions: device use in everyday life and the economics of attention

This thesis investigates smartphone use in naturally occurring contexts with a dataset comprising 200 hours of audio-visual first-person recordings from wearable cameras, and self-confrontation interview video footage (N = 41 users). The situated context in which smartphone use takes place has often been overlooked because of the technical difficulty of capturing context of use, actual action of users, and their subjective experience simultaneously. This research project contributes to filling this gap, with a detailed, mixed-methods analysis of over a thousand individual phone engagement behaviours (EB). We observe that (a) the smartphone is a key structuring element in the flow of daily activities. Participants report complex strategies on how they manage engaging with or avoiding their devices. (b) Unexpectedly, we find that the majority of EB (89%) are initiated by users, not devices; users engage with the phone roughly every five minutes regardless of the context they are in. (c) A large portion of EB seems to stem from contextual cues and an unconscious urge to pick up the device, even when there is no clear reason to do so. d) Participants are surprised about, and often unhappy with how frequently they mindlessly reach for the phone. Our in-depth analysis unveils several overlapping layers of motivations and triggers driving EB. Monitoring incoming notifications, managing time use, responding to social pressures, actually completing a task with the phone, design factors, unconscious urges, as well as the accessibility of the device, and most importantly its affordance for distraction all contribute to picking up the phone. This user drive for EB is used by providers to feed the attention economy. So far, keeping the smartphone outside of the visual field and immediate reach has appeared as the only efficient strategy to prevent overuse

Quantifying Quality of Life

Describes technological methods and tools for objective and quantitative assessment of QoL Appraises technology-enabled methods for incorporating QoL measurements in medicine Highlights the success factors for adoption and scaling of technology-enabled methods This open access book presents the rise of technology-enabled methods and tools for objective, quantitative assessment of Quality of Life (QoL), while following the WHOQOL model. It is an in-depth resource describing and examining state-of-the-art, minimally obtrusive, ubiquitous technologies. Highlighting the required factors for adoption and scaling of technology-enabled methods and tools for QoL assessment, it also describes how these technologies can be leveraged for behavior change, disease prevention, health management and long-term QoL enhancement in populations at large. Quantifying Quality of Life: Incorporating Daily Life into Medicine fills a gap in the field of QoL by providing assessment methods, techniques and tools. These assessments differ from the current methods that are now mostly infrequent, subjective, qualitative, memory-based, context-poor and sparse. Therefore, it is an ideal resource for physicians, physicians in training, software and hardware developers, computer scientists, data scientists, behavioural scientists, entrepreneurs, healthcare leaders and administrators who are seeking an up-to-date resource on this subject

Assessing the Role of Faith-Based Organizations in Health Promotion

Objective: The objective of this analysis was to evaluate the relationship between organizational readiness (OR) and weight loss and physical activity outcomes among faith-based organizations (FBOs). Methods: Data for this study were collected in two phases. Phase I data were based on a feasibility study and targeted African-American individuals (N = 55) who participated in an obesity prevention program. The intervention was accomplished in two stages, which included a 10-week core period followed by a 6-month maintenance period. Phase II data were based on key informant interviews that were conducted with community health leaders (CHLs) (N=6) from participating FBOs. These interviews addressed six dimensions of readiness, with each dimension receiving an independent score that ranged from 1 to 9 (no awareness to a high level of community ownership). Dimension scores were averaged and each FBO was assigned a numerical OR score. OR scores were computed from the interview data, utilizing anchored rating scales outlined in the Community Readiness Model (CRM). These scores were subsequently combined with the Phase I data and used to statistically estimate the associations of OR. Linear mixed models, using SAS/STAT® software, were used to evaluate the relationship between OR scores and weight loss and physical activity while adjusting for covariates. A qualitative analysis of the Phase II data was also performed. Results: Approximately 12.5% of the sample had an OR score of 4, 69.6% had an OR score of 5, while 17.9% were assigned an OR score of 7. An OR score of four indicated a pre-planning stage of readiness. Those with an OR score of 5 were in the preparation stage of readiness, while those with a score of seven were in the stabilization stage. An OR score of 5 was associated with a significant increase in weight (2.532, p=0.048) when compared with an OR score of 7. Post hoc analysis revealed significant mean differences in weight when comparing congregation 1 with congregations 4 (difference=3.452, p=0.016) and 5(difference= 4.646, p=0.0005). Congregation 2 had a significant mean difference in weight compared to both congregations 4 (difference= 5.264, p\u3c0.0006) and 5 (difference= 6.457, p\u3c0.0001). During the maintenance period, Group(s) with an OR score of 5 gained weight compared to those with an OR score of 7 (6.093, p=0.0018). Post hoc analyses revealed significant mean differences between congregation 1 and congregations 4 (difference=7.896, p=0.001) and 5 (difference= 10.708, p=0.003). The mean activity level of group(s) with an OR score of 4 and 5 were 166.02 minutes (p\u3c0.0001) and 177.33 minutes (p\u3c0.001) lower, respectively, than the group with an OR score of 7. Post hoc analysis revealed significant mean differences in physical activity minutes for congregation 1 compared with congregations 2 (difference= -91.698, p=0.011), and 4 (difference= -203.90, p\u3c0.0001). There were also significant mean differences between congregations 2 versus 4 (-112.20, p= 0.010) and 4 versus 5 (155.18, p\u3c.0006). There were also statistically significant differences in physical activity by OR category. Physical activity minutes among group(s) with an OR score of 4 (-1284.21, p\u3c0.0001) and 5 (-933.21, p\u3c0.0001) were lower than those with a score of 7. The post hoc analysis revealed significant mean differences between congregations 2 (-2191.82, p\u3c0.0001), and 4 (-1631.77, p\u3c0.0001) when compared with congregation 1. There were also significant mean differences in physical activity between congregations 2 versus 3 (difference= 2557.60, p=0.0009) and 5(difference=1602.11, p\u3c0.001), 3 versus 4 (difference= -1997.55, p=0.007) and 4 versus 5 (difference= 1042.06, p\u3c0.001). Despite the enthusiasm of participating FBO, the qualitative evaluation revealed that health behavior change can be difficult to adopt and maintain. Moving into an organization that is well structured is seemingly a great formula for success; however, the strength of the organization alone is not sufficient to promote and support health behavior change. Irrespective of an organization’s position on the readiness continuum, several barriers may exist. Primary obstacles included: age of the congregation, competing activities, time frame of the initiative, recognizing the issue and appropriate problem solving, motivation and cost of healthy food options. Conclusion: Although physical activity and weight outcomes were associated with OR scores, the post-hoc analysis revealed variations in outcomes by congregation. Congregational differences may be attributable to intra-group distinctions rather than organizational readiness levels. Therefore, health promotion coordinators must work closely with FBO to pinpoint effective recruitment, implementation, and maintenance strategies that reach the community at various sectors

“Time: A Kaleidoscopic Image of Bermuda’s Sacred Financial Phenomenon and the Wealth of Social-Environmental Diversity”

Michelle’s thesis explores the extent to which a researcher could contribute to change by engaging leaders in conversations that might intensify commitment to or the direction of their actions around socio-environmental decline in Bermuda as a country historically organised in the tradition of an entrepreneurial for-profit enterprise. The framing of a space to reflect on highlighted the significance of time that led to the bricolage design of a heuristic device called a moon gate. Time, the keystone of the moon gate, created a prism lighting up the twists and turns of kaleidoscopic images of the chaos of the markets and the creative diversity of human and nonhuman participants. She undertook a philosophical approach with her study of the contested space of time around sensitivity to maintaining a fulcrum point based on the wellbeing of people and planet