Interaction Paradigms for Brain-Body Interfaces for Computer Users with Brain Injuries

In comparison to all types of injury, those to the brain are among the most likely to result in death or permanent disability. Some of these brain-injured people cannot communicate, recreate, or control their environment due to severe motor impairment. This group of individuals with severe head injury have received limited help from assistive technology. Brain-Computer Interfaces have opened up a spectrum of assistive technologies, which are particularly appropriate for people with traumatic brain injury, especially those who suffer from “locked-in” syndrome. The research challenge here is to develop novel interaction paradigms that suit brain-injured individuals, who could then use it for everyday communications. The developed interaction paradigms should require minimum training, reconfigurable and minimum effort to use. This thesis reports on the development of novel interaction paradigms for Brain-Body Interfaces to help brain-injured people to communicate better, recreate and control their environment using computers despite the severity of their brain injury. The investigation was carried out in three phases. Phase one was an exploratory study where a first novel interaction paradigm was developed and evaluated with able-bodied and disabled participants. Results obtained were fed into the next phase of the investigation. Phase two was carried out with able participants who acted as development group for the second novel interaction paradigm. This second novel interaction paradigm was evaluated with non-verbal participants with severe brain injury in phase three. An iterative design research methodology was chosen to develop the interaction paradigms. A non-invasive assistive technology device named Cyberlink™ was chosen as the Brain-Body Interface. This research improved previous work in this area by developing new interaction paradigms of personalised tiling and discrete acceleration in Brain- Body Interfaces. The research hypothesis of this study ‘that the performance of the Brain-Body Interface can be improved by the use of novel interaction paradigms’ was successfully demonstrated

Seeing the woods for the trees: the problem of information inefficiency and information overload on operator performance

One of the recurring questions in designing dynamic control environments is whether providing more information leads to better operational decisions. The idea of having every piece of information is increasingly tempting (and in safety critical domains often mandatory) but has become a potential obstacle for designers and operators. The present research study examined this challenge of appropriate information design and usability within a railway control setting. A laboratory study was conducted to investigate the presentation of different levels of information (taken from data processing framework, Dadashi et al., 2014) and the association with, and potential prediction of, the performance of a human operator when completing a cognitively demanding problem solving scenario within railways. Results indicated that presenting users only with information corresponding to their cognitive task, and in the absence of other, non task-relevant information, improves the performance of their problem solving/alarm handling. Knowing the key features of interest to various agents (machine or human) and using the data processing framework to guide the optimal level of information required by each of these agents could potentially lead to safer and more usable designs

Shaped by Design "How User-Interface Design Influences Medical Decision Making: The Role of Monitoring Equipment in Anesthetic Practice"

abstract: Objective: The aim of this research is to uncover, via a comprehensive cross study analysis, data patterns that could potentially point to a positive correlation between two main variables: anesthetic monitoring equipment and anesthetic decision making. Of particular interest is the equipment's monitor screen and the extent to which its user interface design influences anesthetic situation awareness (SA) and hence, decision making. It is hypothesized that poor anesthetic diagnosis from inadequate SA may be largely attributable to patient data displays lacking in human factors design considerations. Methods: A systematic search was conducted of existing empirical studies pertaining to patient physiologic monitoring that spanned across interrelated domains, namely, ergonomics, medical informatics, visual computing, cognitive psychology, human factors, clinical monitoring, intensive care medicine, and intelligent systems etc. all published in scholarly research journals between 1970 to August 2012. Anesthetic-related keywords were queried i.e. anesthetic mishaps, patient physiological data displays, anesthetic vigilance etc. (found in Appendix A). This approach yielded a few thousand results, of which 65 empirical studies were pulled. Further extraction of articles having direct connection to the use of data displays within the anesthetic context produced a total of 20 empirical studies. These studies were grouped under two broad categories of Monitoring and Monitors whereby factors directly contributing to the studies' results were identified with the aim to find emerging themes that provide insights involving interface design and medical decision making. Results: There is a direct correlation between user-interface design and decision making. The situation awareness (SA) required for decision making heavily relies upon data displays oriented towards information extraction and integration. In the systematic assessment of empirical studies, it is undeniable how strikingly prominent visual attributes show up as contributing factors to subjects' enhanced performance in the studies. Conclusions: How and to what users direct their perceptual and cognitive resources necessarily influence their perception of the environment, and by extension, their development of situation awareness (SA). Although patient monitoring equipment employed in anesthetic practice has proven to be indispensable in quality patient care, graphical representations of patient data is still far from optimal in the clinical setting. User-interfaces that lend decision support to facilitate SA and subsequent decision making is critical in crisis management.Dissertation/ThesisM.S.D. Design 201

Workload-aware systems and interfaces for cognitive augmentation

In today's society, our cognition is constantly influenced by information intake, attention switching, and task interruptions. This increases the difficulty of a given task, adding to the existing workload and leading to compromised cognitive performances. The human body expresses the use of cognitive resources through physiological responses when confronted with a plethora of cognitive workload. This temporarily mobilizes additional resources to deal with the workload at the cost of accelerated mental exhaustion. We predict that recent developments in physiological sensing will increasingly create user interfaces that are aware of the user’s cognitive capacities, hence able to intervene when high or low states of cognitive workload are detected. In this thesis, we initially focus on determining opportune moments for cognitive assistance. Subsequently, we investigate suitable feedback modalities in a user-centric design process which are desirable for cognitive assistance. We present design requirements for how cognitive augmentation can be achieved using interfaces that sense cognitive workload. We then investigate different physiological sensing modalities to enable suitable real-time assessments of cognitive workload. We provide empirical evidence that the human brain is sensitive to fluctuations in cognitive resting states, hence making cognitive effort measurable. Firstly, we show that electroencephalography is a reliable modality to assess the mental workload generated during the user interface operation. Secondly, we use eye tracking to evaluate changes in eye movements and pupil dilation to quantify different workload states. The combination of machine learning and physiological sensing resulted in suitable real-time assessments of cognitive workload. The use of physiological sensing enables us to derive when cognitive augmentation is suitable. Based on our inquiries, we present applications that regulate cognitive workload in home and work settings. We deployed an assistive system in a field study to investigate the validity of our derived design requirements. Finding that workload is mitigated, we investigated how cognitive workload can be visualized to the user. We present an implementation of a biofeedback visualization that helps to improve the understanding of brain activity. A final study shows how cognitive workload measurements can be used to predict the efficiency of information intake through reading interfaces. Here, we conclude with use cases and applications which benefit from cognitive augmentation. This thesis investigates how assistive systems can be designed to implicitly sense and utilize cognitive workload for input and output. To do so, we measure cognitive workload in real-time by collecting behavioral and physiological data from users and analyze this data to support users through assistive systems that adapt their interface according to the currently measured workload. Our overall goal is to extend new and existing context-aware applications by the factor cognitive workload. We envision Workload-Aware Systems and Workload-Aware Interfaces as an extension in the context-aware paradigm. To this end, we conducted eight research inquiries during this thesis to investigate how to design and create workload-aware systems. Finally, we present our vision of future workload-aware systems and workload-aware interfaces. Due to the scarce availability of open physiological data sets, reference implementations, and methods, previous context-aware systems were limited in their ability to utilize cognitive workload for user interaction. Together with the collected data sets, we expect this thesis to pave the way for methodical and technical tools that integrate workload-awareness as a factor for context-aware systems.Tagtäglich werden unsere kognitiven Fähigkeiten durch die Verarbeitung von unzähligen Informationen in Anspruch genommen. Dies kann die Schwierigkeit einer Aufgabe durch mehr oder weniger Arbeitslast beeinflussen. Der menschliche Körper drückt die Nutzung kognitiver Ressourcen durch physiologische Reaktionen aus, wenn dieser mit kognitiver Arbeitsbelastung konfrontiert oder überfordert wird. Dadurch werden weitere Ressourcen mobilisiert, um die Arbeitsbelastung vorübergehend zu bewältigen. Wir prognostizieren, dass die derzeitige Entwicklung physiologischer Messverfahren kognitive Leistungsmessungen stets möglich machen wird, um die kognitive Arbeitslast des Nutzers jederzeit zu messen. Diese sind in der Lage, einzugreifen wenn eine zu hohe oder zu niedrige kognitive Belastung erkannt wird. Wir konzentrieren uns zunächst auf die Erkennung passender Momente für kognitive Unterstützung welche sich der gegenwärtigen kognitiven Arbeitslast bewusst sind. Anschließend untersuchen wir in einem nutzerzentrierten Designprozess geeignete Feedbackmechanismen, die zur kognitiven Assistenz beitragen. Wir präsentieren Designanforderungen, welche zeigen wie Schnittstellen eine kognitive Augmentierung durch die Messung kognitiver Arbeitslast erreichen können. Anschließend untersuchen wir verschiedene physiologische Messmodalitäten, welche Bewertungen der kognitiven Arbeitsbelastung in Realzeit ermöglichen. Zunächst validieren wir empirisch, dass das menschliche Gehirn auf kognitive Arbeitslast reagiert. Es zeigt sich, dass die Ableitung der kognitiven Arbeitsbelastung über Elektroenzephalographie eine geeignete Methode ist, um den kognitiven Anspruch neuartiger Assistenzsysteme zu evaluieren. Anschließend verwenden wir Eye-Tracking, um Veränderungen in den Augenbewegungen und dem Durchmesser der Pupille unter verschiedenen Intensitäten kognitiver Arbeitslast zu bewerten. Das Anwenden von maschinellem Lernen führt zu zuverlässigen Echtzeit-Bewertungen kognitiver Arbeitsbelastung. Auf der Grundlage der bisherigen Forschungsarbeiten stellen wir Anwendungen vor, welche die Kognition im häuslichen und beruflichen Umfeld unterstützen. Die physiologischen Messungen stellen fest, wann eine kognitive Augmentierung sich als günstig erweist. In einer Feldstudie setzen wir ein Assistenzsystem ein, um die erhobenen Designanforderungen zur Reduktion kognitiver Arbeitslast zu validieren. Unsere Ergebnisse zeigen, dass die Arbeitsbelastung durch den Einsatz von Assistenzsystemen reduziert wird. Im Anschluss untersuchen wir, wie kognitive Arbeitsbelastung visualisiert werden kann. Wir stellen eine Implementierung einer Biofeedback-Visualisierung vor, die das Nutzerverständnis zum Verlauf und zur Entstehung von kognitiver Arbeitslast unterstützt. Eine abschließende Studie zeigt, wie Messungen kognitiver Arbeitslast zur Vorhersage der aktuellen Leseeffizienz benutzt werden können. Wir schließen hierbei mit einer Reihe von Applikationen ab, welche sich kognitive Arbeitslast als Eingabe zunutze machen. Die vorliegende wissenschaftliche Arbeit befasst sich mit dem Design von Assistenzsystemen, welche die kognitive Arbeitslast der Nutzer implizit erfasst und diese bei der Durchführung alltäglicher Aufgaben unterstützt. Dabei werden physiologische Daten erfasst, um Rückschlüsse in Realzeit auf die derzeitige kognitive Arbeitsbelastung zu erlauben. Anschließend werden diese Daten analysiert, um dem Nutzer strategisch zu assistieren. Das Ziel dieser Arbeit ist die Erweiterung neuartiger und bestehender kontextbewusster Benutzerschnittstellen um den Faktor kognitive Arbeitslast. Daher werden in dieser Arbeit arbeitslastbewusste Systeme und arbeitslastbewusste Benutzerschnittstellen als eine zusätzliche Dimension innerhalb des Paradigmas kontextbewusster Systeme präsentiert. Wir stellen acht Forschungsstudien vor, um die Designanforderungen und die Implementierung von kognitiv arbeitslastbewussten Systemen zu untersuchen. Schließlich stellen wir unsere Vision von zukünftigen kognitiven arbeitslastbewussten Systemen und Benutzerschnittstellen vor. Durch die knappe Verfügbarkeit öffentlich zugänglicher Datensätze, Referenzimplementierungen, und Methoden, waren Kontextbewusste Systeme in der Auswertung kognitiver Arbeitslast bezüglich der Nutzerinteraktion limitiert. Ergänzt durch die in dieser Arbeit gesammelten Datensätze erwarten wir, dass diese Arbeit den Weg für methodische und technische Werkzeuge ebnet, welche kognitive Arbeitslast als Faktor in das Kontextbewusstsein von Computersystemen integriert

ESMD Space Grant Faculty Report

The strength of the Exploration Systems Mission Directorate ESMD Faculty Project lies in its ability to meet National Aeronautics Space Administration NASA's Strategic Educational Outcome 1 by developing a sustainable and long-term integration of student involvement at academic institutions with all NASA Centers. This outcome is achieved by a three-fold approach: 1) by collecting Senior Design projects pertaining to Constellation work performed at each of the ten NASA Centers, 2) by engaging students at Minority Serving Institutions in the art of systems engineering and systems design of technologies required for space exploration, and 3) by identifying potential internships at each Center relative to exploration that provide students who are supported by their institutional Space Grant to engage in on-going mission-level and explorative systems designs. The objectives of the ESMD Faculty Project are to: 1. Aid the Centers (both Education Offices and associated technical organizations) in providing relevant opportunities for the ESMD Space Grant Program to support student and faculty in Senior Design projects 2. Enable better matches between the ESMD work required and what the Space Grant Consortia can do to effectively contribute to NASA programs 3. Provide the Space Grant Consortia an opportunity to strengthen relations with the NASA Centers 4. Develop better collective understanding of the U.S. Space Exploration Policy by the Center, Space Grant, faculty, Education Office, and students 5. Enable Space Grant institution faculty to better prepare their students to meet current and future NASA needs 6. Enable the Center Education Offices to strengthen their ties to their technical organizations and Space Grant Consortia 7. Aid KSC in gaining a greater and more detailed understanding of each of the Center activities Senior Design projects are intended to stimulate undergraduate students on current NASA activities related to lunar, Mars, and other planetary missions and to bring out innovative and novel ideas that can be used to complement those currently under development at respective NASA Centers. Additionally, such academic involvement would better the prospects for graduating seniors to pursue graduate studies and to seek careers in the space industry with a strong sense for systems engineering and understanding of design concepts. Internships, on the other hand, are intended to provide hands-on experience to students by engaging them in diverse state-of-the-art technology development, prototype bread-boarding, computer modeling and simulations, hardware and software testing, and other activities that provide students a strong perspective of NASA's vision and mission in enhancing the knowledge of Earth and space planetary sciences. Ten faculty members, each from a Space Grant Consortium-affiliated university, worked at ten NASA Centers for five weeks between June 2 and July 3, 2008. The project objectives listed above were achieved. In addition to collecting data on Senior Design ideas and identifying possible internships that would benefit NASA/ESMD, the faculty fellows promoted and collected data when required for other ESMD-funded programs and helped the Center's Education Office, as,needed.

Interaction paradigms for brain-body interfaces for computer users with brain injuries

In comparison to all types of injury, those to the brain are among the most likely to result in death or permanent disability. Some of these brain-injured people cannot communicate, recreate, or control their environment due to severe motor impairment. This group of individuals with severe head injury have received limited help from assistive technology. Brain-Computer Interfaces have opened up a spectrum of assistive technologies, which are particularly appropriate for people with traumatic brain injury, especially those who suffer from “locked-in” syndrome. The research challenge here is to develop novel interaction paradigms that suit brain-injured individuals, who could then use it for everyday communications. The developed interaction paradigms should require minimum training, reconfigurable and minimum effort to use. This thesis reports on the development of novel interaction paradigms for Brain-Body Interfaces to help brain-injured people to communicate better, recreate and control their environment using computers despite the severity of their brain injury. The investigation was carried out in three phases. Phase one was an exploratory study where a first novel interaction paradigm was developed and evaluated with able-bodied and disabled participants. Results obtained were fed into the next phase of the investigation. Phase two was carried out with able participants who acted as development group for the second novel interaction paradigm. This second novel interaction paradigm was evaluated with non-verbal participants with severe brain injury in phase three. An iterative design research methodology was chosen to develop the interaction paradigms. A non-invasive assistive technology device named Cyberlink™ was chosen as the Brain-Body Interface. This research improved previous work in this area by developing new interaction paradigms of personalised tiling and discrete acceleration in Brain- Body Interfaces. The research hypothesis of this study ‘that the performance of the Brain-Body Interface can be improved by the use of novel interaction paradigms’ was successfully demonstrated.EThOS - Electronic Theses Online ServiceGBUnited Kingdo