Workload-aware systems and interfaces for cognitive augmentation

In today's society, our cognition is constantly influenced by information intake, attention switching, and task interruptions. This increases the difficulty of a given task, adding to the existing workload and leading to compromised cognitive performances. The human body expresses the use of cognitive resources through physiological responses when confronted with a plethora of cognitive workload. This temporarily mobilizes additional resources to deal with the workload at the cost of accelerated mental exhaustion. We predict that recent developments in physiological sensing will increasingly create user interfaces that are aware of the user’s cognitive capacities, hence able to intervene when high or low states of cognitive workload are detected. In this thesis, we initially focus on determining opportune moments for cognitive assistance. Subsequently, we investigate suitable feedback modalities in a user-centric design process which are desirable for cognitive assistance. We present design requirements for how cognitive augmentation can be achieved using interfaces that sense cognitive workload. We then investigate different physiological sensing modalities to enable suitable real-time assessments of cognitive workload. We provide empirical evidence that the human brain is sensitive to fluctuations in cognitive resting states, hence making cognitive effort measurable. Firstly, we show that electroencephalography is a reliable modality to assess the mental workload generated during the user interface operation. Secondly, we use eye tracking to evaluate changes in eye movements and pupil dilation to quantify different workload states. The combination of machine learning and physiological sensing resulted in suitable real-time assessments of cognitive workload. The use of physiological sensing enables us to derive when cognitive augmentation is suitable. Based on our inquiries, we present applications that regulate cognitive workload in home and work settings. We deployed an assistive system in a field study to investigate the validity of our derived design requirements. Finding that workload is mitigated, we investigated how cognitive workload can be visualized to the user. We present an implementation of a biofeedback visualization that helps to improve the understanding of brain activity. A final study shows how cognitive workload measurements can be used to predict the efficiency of information intake through reading interfaces. Here, we conclude with use cases and applications which benefit from cognitive augmentation. This thesis investigates how assistive systems can be designed to implicitly sense and utilize cognitive workload for input and output. To do so, we measure cognitive workload in real-time by collecting behavioral and physiological data from users and analyze this data to support users through assistive systems that adapt their interface according to the currently measured workload. Our overall goal is to extend new and existing context-aware applications by the factor cognitive workload. We envision Workload-Aware Systems and Workload-Aware Interfaces as an extension in the context-aware paradigm. To this end, we conducted eight research inquiries during this thesis to investigate how to design and create workload-aware systems. Finally, we present our vision of future workload-aware systems and workload-aware interfaces. Due to the scarce availability of open physiological data sets, reference implementations, and methods, previous context-aware systems were limited in their ability to utilize cognitive workload for user interaction. Together with the collected data sets, we expect this thesis to pave the way for methodical and technical tools that integrate workload-awareness as a factor for context-aware systems.Tagtäglich werden unsere kognitiven Fähigkeiten durch die Verarbeitung von unzähligen Informationen in Anspruch genommen. Dies kann die Schwierigkeit einer Aufgabe durch mehr oder weniger Arbeitslast beeinflussen. Der menschliche Körper drückt die Nutzung kognitiver Ressourcen durch physiologische Reaktionen aus, wenn dieser mit kognitiver Arbeitsbelastung konfrontiert oder überfordert wird. Dadurch werden weitere Ressourcen mobilisiert, um die Arbeitsbelastung vorübergehend zu bewältigen. Wir prognostizieren, dass die derzeitige Entwicklung physiologischer Messverfahren kognitive Leistungsmessungen stets möglich machen wird, um die kognitive Arbeitslast des Nutzers jederzeit zu messen. Diese sind in der Lage, einzugreifen wenn eine zu hohe oder zu niedrige kognitive Belastung erkannt wird. Wir konzentrieren uns zunächst auf die Erkennung passender Momente für kognitive Unterstützung welche sich der gegenwärtigen kognitiven Arbeitslast bewusst sind. Anschließend untersuchen wir in einem nutzerzentrierten Designprozess geeignete Feedbackmechanismen, die zur kognitiven Assistenz beitragen. Wir präsentieren Designanforderungen, welche zeigen wie Schnittstellen eine kognitive Augmentierung durch die Messung kognitiver Arbeitslast erreichen können. Anschließend untersuchen wir verschiedene physiologische Messmodalitäten, welche Bewertungen der kognitiven Arbeitsbelastung in Realzeit ermöglichen. Zunächst validieren wir empirisch, dass das menschliche Gehirn auf kognitive Arbeitslast reagiert. Es zeigt sich, dass die Ableitung der kognitiven Arbeitsbelastung über Elektroenzephalographie eine geeignete Methode ist, um den kognitiven Anspruch neuartiger Assistenzsysteme zu evaluieren. Anschließend verwenden wir Eye-Tracking, um Veränderungen in den Augenbewegungen und dem Durchmesser der Pupille unter verschiedenen Intensitäten kognitiver Arbeitslast zu bewerten. Das Anwenden von maschinellem Lernen führt zu zuverlässigen Echtzeit-Bewertungen kognitiver Arbeitsbelastung. Auf der Grundlage der bisherigen Forschungsarbeiten stellen wir Anwendungen vor, welche die Kognition im häuslichen und beruflichen Umfeld unterstützen. Die physiologischen Messungen stellen fest, wann eine kognitive Augmentierung sich als günstig erweist. In einer Feldstudie setzen wir ein Assistenzsystem ein, um die erhobenen Designanforderungen zur Reduktion kognitiver Arbeitslast zu validieren. Unsere Ergebnisse zeigen, dass die Arbeitsbelastung durch den Einsatz von Assistenzsystemen reduziert wird. Im Anschluss untersuchen wir, wie kognitive Arbeitsbelastung visualisiert werden kann. Wir stellen eine Implementierung einer Biofeedback-Visualisierung vor, die das Nutzerverständnis zum Verlauf und zur Entstehung von kognitiver Arbeitslast unterstützt. Eine abschließende Studie zeigt, wie Messungen kognitiver Arbeitslast zur Vorhersage der aktuellen Leseeffizienz benutzt werden können. Wir schließen hierbei mit einer Reihe von Applikationen ab, welche sich kognitive Arbeitslast als Eingabe zunutze machen. Die vorliegende wissenschaftliche Arbeit befasst sich mit dem Design von Assistenzsystemen, welche die kognitive Arbeitslast der Nutzer implizit erfasst und diese bei der Durchführung alltäglicher Aufgaben unterstützt. Dabei werden physiologische Daten erfasst, um Rückschlüsse in Realzeit auf die derzeitige kognitive Arbeitsbelastung zu erlauben. Anschließend werden diese Daten analysiert, um dem Nutzer strategisch zu assistieren. Das Ziel dieser Arbeit ist die Erweiterung neuartiger und bestehender kontextbewusster Benutzerschnittstellen um den Faktor kognitive Arbeitslast. Daher werden in dieser Arbeit arbeitslastbewusste Systeme und arbeitslastbewusste Benutzerschnittstellen als eine zusätzliche Dimension innerhalb des Paradigmas kontextbewusster Systeme präsentiert. Wir stellen acht Forschungsstudien vor, um die Designanforderungen und die Implementierung von kognitiv arbeitslastbewussten Systemen zu untersuchen. Schließlich stellen wir unsere Vision von zukünftigen kognitiven arbeitslastbewussten Systemen und Benutzerschnittstellen vor. Durch die knappe Verfügbarkeit öffentlich zugänglicher Datensätze, Referenzimplementierungen, und Methoden, waren Kontextbewusste Systeme in der Auswertung kognitiver Arbeitslast bezüglich der Nutzerinteraktion limitiert. Ergänzt durch die in dieser Arbeit gesammelten Datensätze erwarten wir, dass diese Arbeit den Weg für methodische und technische Werkzeuge ebnet, welche kognitive Arbeitslast als Faktor in das Kontextbewusstsein von Computersystemen integriert

Innovations in the Food System: Exploring the Future of Food

Innovations in Food Systems should be: Inclusive: ensuring economic and social inclusion for all food system actors, especially smallholders, women, and youth; Sustainable: minimizing negative environmental impacts, conserving scarce natural resources, and strengthening resiliency against future shocks; Efficient: producing adequate quantities of food for global needs while minimizing postharvest loss and consumer waste; Nutritious and healthy: enabling the consumption of a diverse range of healthy, nutritious, and safe foods. These are ambitious goals that will require multidisciplinary effort—from engineering to life sciences, biotechnology, medical sciences, social sciences, and economic sciences. New technologies and scientific discoveries are the solutions to the increasing demand for sufficient, safe, healthy, and sustainable foods influenced by the increased public awareness of their importance

Surveillance : an investigation into individual reactions to the capture and recording of personal images through interpretation of the moment

This research project explores surveillance and how individuals react to its presence. This is achieved through research into the development of surveillance, exploring early techniques such as espionage as discussed by Crowdy, (2006), and Bentham’s Panopticon. There is a focus on how early examples of surveillance evolved into and influenced techniques and technologies in use today. Further investigation into privacy takes place and its relationship with modern surveillance is explored.In order to achieve the research goal, that is, an investigation into how targeted subjects react to being surveilled, research and critical analysis into surveillance art and installations was completed and used to aid in the design of a practical investigation. In addition, the practical element of the study provided the opportunity for participants to complete questionnaires, thus enabling analysis of responses that demonstrate their understanding and views on surveillance:Collected images and data were analyzed in order to address the research question: Can we, through a surveillance piece, obtain a critical response to individual reactions to ‘being viewed’? Additional work involved investigating if a pre-held conception had any link to the outcome of the practical research.The captured images and interpretation of those images proved successful, but this process is interpretive and in no way conclusive. Bearing in mind the subjectivity of any analysis, there is some indication that the practical experience has an impact on the expressed views of participants

Enhancing Geospatial Data: Collecting and Visualising User-Generated Content Through Custom Toolkits and Cloud Computing Workflows

Through this thesis we set the hypothesis that, via the creation of a set of custom toolkits, using cloud computing, online user-generated content, can be extracted from emerging large-scale data sets, allowing the collection, analysis and visualisation of geospatial data by social scientists. By the use of a custom-built suite of software, known as the ‘BigDataToolkit’, we examine the need and use of cloud computing and custom workflows to open up access to existing online data as well as setting up processes to enable the collection of new data. We examine the use of the toolkit to collect large amounts of data from various online sources, such as Social Media Application Programming Interfaces (APIs) and data stores, to visualise the data collected in real-time. Through the execution of these workflows, this thesis presents an implementation of a smart collector framework to automate the collection process to significantly increase the amount of data that can be obtained from the standard API endpoints. By the use of these interconnected methods and distributed collection workflows, the final system is able to collect and visualise a larger amount of data in real time than single system data collection processes used within traditional social media analysis. Aimed at allowing researchers without a core understanding of the intricacies of computer science, this thesis provides a methodology to open up new data sources to not only academics but also wider participants, allowing the collection of user-generated geographic and textual content, en masse. A series of case studies are provided, covering applications from the single researcher collecting data through to collection via the use of televised media. These are examined in terms of the tools created and the opportunities opened, allowing real-time analysis of data, collected via the use of the developed toolkit

Towards Socially Integrative Cities

This book is based on the results of the TRANSURBAN-EU-CHINA project. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 770141. The material presented in this book reflects only the authors' views. It does not reflect the official opinion of the European Commission. The TRANS-URBAN-EU-CHINA project or the European Commission are not liable for the contents of the chapters or any use that may be made of the information contained therein. About The book Towards Socially Integrative Cities deals with the transition towards urban sustainability through socially integrative cities in Europe and China. It shares the impactful original research results of an EU-funded R&I project involving eight European and six Chinese partners (TRANS-URBAN-EU-CHINA; see www.transurbaneuchina.eu). Three each of the six editors come from European and Chinese universities and research institutions. The first articles (No. 1–4) provide an overview and insights into the conceptual basis of the book. The socially integrative city is framed by discussions in academia and practice, and it is defined in a comprehensive way as an element of urban sustainability. Sustainability transitions in China are linked to international discussions and the challenges are articulated. Moreover, the discussions touch on the strengths and weaknesses related to managing urban growth and the rapid expansion of cities. Land development tools are discussed with regard to the opportunities they offer for creating socially integrative cities. A second set of articles (No. 5–9) focus on socially integrative urban regeneration of cities. After an overview of policies and strategies in Europe and China, detailed aspects are discussed, such as community building through public engagement, challenges of place-making, and the role of education and life-long learning. Finally, a manuscript on heritage preservation and its impact on social integration in urban regeneration concludes this section. A third set of articles (No. 10–13) look into issues of the transformation of cities and sustainability transitions. Transformation is understood as a complex set of interactions. The development of tools, such as community platforms for information and dialogue transfer, are discussed as an instrument to facilitate transition processes. The transformative capacity of cities in Europe and China is discussed in an article that looks into possibilities to narrow the gaps between urban planning and development. Smart and eco-cities in Europe and China are taken as an empirical base. Advanced methods, such as the Social Cost–Benefit Analysis, may support social integration. Finally, an example of how the use of multiple data sources can speed up the digital transition in cities and provide decision support for social integration is presented. The final set of articles (No. 14 and 15) deal with questions of replicability of experiences and the role of concrete urban experiments in so-called Urban Living Laboratories. Methods to explore the replication potential of urban solutions for socially integrative cities are discussed. Finally, the potential of urban living laboratories for nurturing open urban innovation in Chinese cities is scrutinized. Several examples are discussed, and conclusions regarding the enhancement of social integration in cities are drawn. The book is edited by Bernhard Müller (Technische Universität Dresden, TUD), Baojun Yang (China Academy of Urban Planning and Design, CAUPD), Liu Jian (Tsinghua University), Jianming Cai (China Academy of Sciences, CAS), Paulina Schiappacasse (TUD) and Hans-Martin Neumann (Austrian Institute of Technology, AIT)