An Unobtrusive System to Measure, Assess, and Predict Cognitive Workload in Real-World Environments

Across many careers, individuals face alternating periods of high and low attention and cognitive workload, which can result in impaired cognitive functioning and can be detrimental to job performance. For example, some professions (e.g., fire fighters, emergency medical personnel, doctors and nurses working in an emergency room, pilots) require long periods of low workload (boredom), followed by sudden, high-tempo operations during which they may be required to respond to an emergency and perform at peak cognitive levels. Conversely, other professions (e.g., air traffic controllers, market investors in financial industries, analysts) require long periods of high workload and multitasking during which the addition of just one more task results in cognitive overload resulting in mistakes. An unobtrusive system to measure, assess, and predict cognitive workload could warn individuals, their teammates, or their supervisors when steps should be taken to augment cognitive readiness. In this talk I will describe an approach to this problem that we have found to be successful across work domains including: (1) a suite of unobtrusive, field-ready neurophysiological, physiological, and behavioral sensors that are chosen to best suit the target environment; (2) custom algorithms and statistical techniques to process and time-align raw data originating from the sensor suite; (3) probabilistic and statistical models designed to interpret the data into the human state of interest (e.g., cognitive workload, attention, fatigue); (4) and machine-learning techniques to predict upcoming performance based on the current pattern of events, and (5) display of each piece of information depending on the needs of the target user who may or may not want to drill down into the functioning of the system to determine how conclusions about human state and performance are determined. I will then focus in on our experimental results from our custom functional near-infrared spectroscopy sensor, designed to operate in real-world environments to be worn comfortably (e.g., positioned into a baseball cap or a surgeons cap) to measure changes in brain blood oxygenation without adding burden to the individual being assessed

Early diagnosis of disorders based on behavioural shifts and biomedical signals

There are many disorders that directly affect people’s behaviour. The people that are suffering from such a disorder are not aware of their situation, and too often the disorders are identified by relatives or co-workers because they notice behavioural shifts. However, when these changes become noticeable, it is often too late and irreversible damages have already been produced. Early detection is the key to prevent severe health-related damages and healthcare costs, as well as to improve people’s quality of life. Nowadays, in full swing of ubiquitous computing paradigm, users’ behaviour patterns can be unobtrusively monitored by means of interactions with many electronic devices. The application of this technology for the problem at hand would lead to the development of systems that are able to monitor disorders’ onset and progress in an ubiquitous and unobtrusive way, thus enabling their early detection. Some attempts for the detection of specific disorders based on these technologies have been proposed, but a global methodology that could be useful for the early detection of a wide range of disorders is still missing. This thesis aims to fill that gap by presenting as main contribution a global screening methodology for the early detection of disorders based on unobtrusive monitoring of physiological and behavioural data. The proposed methodology is the result of a cross-case analysis between two individual validation scenarios: stress in the workplace and Alzheimer’s Disease (AD) at home, from which conclusions that contribute to each of the two research fields have been drawn. The analysis of similarities and differences between the two case studies has led to a complete and generalized definition of the steps to be taken for the detection of a new disorder based on ubiquitous computing.Jendearen portaeran eragin zuzena duten gaixotasun ugari daude. Hala ere, askotan, gaixotasuna pairatzen duten pertsonak ez dira euren egoerataz ohartzen, eta familiarteko edo lankideek identifikatu ohi dute berau jokabide aldaketetaz ohartzean. Portaera aldaketa hauek nabarmentzean, ordea, beranduegi izan ohi da eta atzerazeinak diren kalteak eraginda egon ohi dira. Osasun kalte larriak eta gehiegizko kostuak ekiditeko eta gaixoen bizi kalitatea hobetzeko gakoa, gaixotasuna garaiz detektatzea da. Gaur egun, etengabe zabaltzen ari den Nonahiko Konputazioaren paradigmari esker, erabiltzaileen portaera ereduak era diskretu batean monitorizatu daitezke, gailu teknologikoekin izandako interakzioari esker. Eskuartean dugun arazoari konponbidea emateko teknologi hau erabiltzeak gaixotasunen sorrera eta aurrerapena nonahi eta era diskretu batean monitorizatzeko gai diren sistemak garatzea ekarriko luke, hauek garaiz hautematea ahalbidetuz. Gaixotasun konkretu batzuentzat soluzioak proposatu izan dira teknologi honetan oinarrituz, baina metodologia orokor bat, gaixotasun sorta zabal baten detekzio goiztiarrerako erabilgarria izango dena, oraindik ez da aurkeztu. Tesi honek hutsune hori betetzea du helburu, mota honetako gaixotasunak garaiz hautemateko, era diskretu batean atzitutako datu fisiologiko eta konportamentalen erabileran oinarritzen den behaketa sistema orokor bat proposatuz. Proposatutako metodologia bi balidazio egoera desberdinen arteko analisi gurutzatu baten emaitza da: estresa lantokian eta Alzheimerra etxean, balidazio egoera bakoitzari dagozkion ekarpenak ere ondorioztatu ahal izan direlarik. Bi kasuen arteko antzekotasun eta desberdintasunen analisiak, gaixotasun berri bat nonahiko konputazioan oinarrituta detektatzeko jarraitu beharreko pausoak bere osotasunean eta era orokor batean definitzea ahalbidetu du

On driver behavior recognition for increased safety:A roadmap

Advanced Driver-Assistance Systems (ADASs) are used for increasing safety in the automotive domain, yet current ADASs notably operate without taking into account drivers’ states, e.g., whether she/he is emotionally apt to drive. In this paper, we first review the state-of-the-art of emotional and cognitive analysis for ADAS: we consider psychological models, the sensors needed for capturing physiological signals, and the typical algorithms used for human emotion classification. Our investigation highlights a lack of advanced Driver Monitoring Systems (DMSs) for ADASs, which could increase driving quality and security for both drivers and passengers. We then provide our view on a novel perception architecture for driver monitoring, built around the concept of Driver Complex State (DCS). DCS relies on multiple non-obtrusive sensors and Artificial Intelligence (AI) for uncovering the driver state and uses it to implement innovative Human–Machine Interface (HMI) functionalities. This concept will be implemented and validated in the recently EU-funded NextPerception project, which is briefly introduced

Measuring cognitive load and cognition: metrics for technology-enhanced learning

This critical and reflective literature review examines international research published over the last decade to summarise the different kinds of measures that have been used to explore cognitive load and critiques the strengths and limitations of those focussed on the development of direct empirical approaches. Over the last 40 years, cognitive load theory has become established as one of the most successful and influential theoretical explanations of cognitive processing during learning. Despite this success, attempts to obtain direct objective measures of the theory's central theoretical construct – cognitive load – have proved elusive. This obstacle represents the most significant outstanding challenge for successfully embedding the theoretical and experimental work on cognitive load in empirical data from authentic learning situations. Progress to date on the theoretical and practical approaches to cognitive load are discussed along with the influences of individual differences on cognitive load in order to assess the prospects for the development and application of direct empirical measures of cognitive load especially in technology-rich contexts

Smart workplaces: a system proposal for stress management

Over the past last decades of contemporary society, workplaces have become the primary source of many health issues, leading to mental problems such as stress, depression, and anxiety. Among the others, environmental aspects have shown to be the causes of stress, illness, and lack of productivity. With the arrival of new technologies, especially in the smart workplaces field, most studies have focused on investigating the building energy efficiency models and human thermal comfort. However, little has been applied to occupants’ stress recognition and well-being overall. Due to this fact, this present study aims to propose a stress management solution for an interactive design system that allows the adapting of comfortable environmental conditions according to the user preferences by measuring in real-time the environmental and biological characteristics, thereby helping to prevent stress, as well as to enable users to cope stress when being stressed. The secondary objective will focus on evaluating one part of the system: the mobile application. The proposed system uses several usability methods to identify users’ needs, behavior, and expectations from the user-centered design approach. Applied methods, such as User Research, Card Sorting, and Expert Review, allowed us to evaluate the design system according to Heuristics Analysis, resulting in improved usability of interfaces and experience. The study presents the research results, the design interface, and usability tests. According to the User Research results, temperature and noise are the most common environmental stressors among the users causing stress and uncomfortable conditions to work in, and the preference for physical activities over the digital solutions for coping with stress. Additionally, the System Usability Scale (SUS) results identified that the system’s usability was measured as “excellent” and “acceptable” with a final score of 88 points out of the 100. It is expected that these conclusions can contribute to future investigations in the smart workplaces study field and their interaction with the people placed there.Nas últimas décadas da sociedade contemporânea, o local de trabalho tem se tornado principal fonte de muitos problemas de saúde mental, como o stress, depressão e ansiedade. Os aspetos ambientais têm se revelado como as causas de stress, doenças, falta de produtividade, entre outros. Atualmente, com a chegada de novas tecnologias, principalmente na área de locais de trabalho inteligentes, a maioria dos estudos tem se concentrado na investigação de modelos de eficiência energética de edifícios e conforto térmico humano. No entanto, pouco foi aplicado ao reconhecimento do stress dos ocupantes e ao bem-estar geral das pessoas. Diante disso, o objetivo principal é propor um sistema de design de gestão do stress para um sistema de design interativo que permita adaptar as condições ambientais de acordo com as preferências de utilizador, medindo em tempo real as características ambientais e biológicas, auxiliando assim na prevenção de stress, bem como ajuda os utilizadores a lidar com o stress quando estão sob o mesmo. O segundo objetivo é desenhar e avaliar uma parte do projeto — o protótipo da aplicação móvel através da realização de testes de usabilidade. O sistema proposto resulta da abordagem de design centrado no utilizador, utilizando diversos métodos de usabilidade para identificar as necessidades, comportamentos e as expectativas dos utilizadores. Métodos aplicados, como Pesquisa de Usuário, Card Sorting e Revisão de Especialistas, permitiram avaliar o sistema de design de acordo com a análise heurística, resultando numa melhoria na usabilidade das interfaces e experiência. O estudo apresenta os resultados da pesquisa, a interface do design e os testes de usabilidade. De acordo com os resultados de User Research, a temperatura e o ruído são os stressores ambientais mais comuns entre os utilizadores, causando stresse e condições menos favoráveis para trabalhar, igualmente existe uma preferência por atividades físicas sobre as soluções digitais na gestão do stresse. Adicionalmente, os resultados de System Usability Scale (SUS) identificaram a usabilidade do sistema de design como “excelente” e “aceitável” com pontuação final de 88 pontos em 100. É esperado que essas conclusões possam contribuir para futuras investigações no campo de estudo dos smart workplaces e sua interação com os utilizadores

Individual differences in working memory affect situation awareness

2011 Summer.Includes bibliographical references.Situation awareness (SA) is a construct that brings together theories of attention, memory, and expertise in an empirical effort to showcase what awareness is and how it is acquired by operators. Endsley (1995a) defined SA in a way that includes many theoretical associations between awareness and specific memory and attention mechanisms. Work characterizing these relationships has been sparse, however, particularly with regard to the influence of working memory (WM) on SA in novices. An experiment was devised which principally investigated novice SA as a theorized function of WM across two distinct tasks; one in which operator attention and perception (Level 1 SA) was assessed, and one in which an operator's ability to respond to events in the future (Level 3 SA) was implicitly assessed. Factors analysis was used and resulting outcomes from three WM tasks loaded well onto one overall WM factor. Findings from 99 participants indicate that WM does have a correlative and predictive relationship with Level 3, but not Level 1 SA. Results reported here contribute to ongoing theory and experimental work in applied psychology with regard to SA and individual differences, showing WM influences awareness in novice performance even in the case where SA measures are not memory-reliant

Habitability and Performance Issues for Long Duration Space Flights

Advancing technology, coupled with the desire to explore space has resulted in increasingly longer manned space missions. Although the Long Duration Space Flights (LDSF) have provided a considerable amount of scientific research on human ability to function in extreme environments, findings indicate long duration missions take a toll on the individual, both physiologically and psychologically. These physiological and psychological issues manifest themselves in performance decrements; and could lead to serious errors endangering the mission, spacecraft and crew. The purpose of this paper is to document existing knowledge of the effects of LDSF on performance, habitability, and workload and to identify and assess potential tools designed to address these decrements as well as propose an implementation plan to address the habitability, performance and workload issues

Best Practices for Evaluating Flight Deck Interfaces for Transport Category Aircraft with Particular Relevance to Issues of Attention, Awareness, and Understanding CAST SE-210 Output 2 Report 6 of 6

Attention, awareness, and understanding of the flight crew are a critical contributor to safety and the flight deck plays a critical role in supporting these cognitive functions. Changes to the flight deck need to be evaluated for whether the changed device provides adequate support for these functions. This report describes a set of diverse evaluation methods. The report recommends designing the interface-evaluation to span the phases of the device development, from early to late, and it provides methods appropriate at each phase. It describes the various ways in which an interface or interface component can fail to support awareness as potential issues to be assessed in evaluation. It summarizes appropriate methods to evaluate different issues concerning inadequate support for these functions, throughout the phases of development