Methods and techniques for analyzing human factors facets on drivers

Mención Internacional en el título de doctorWith millions of cars moving daily, driving is the most performed activity worldwide. Unfortunately, according to the World Health Organization (WHO), every year, around 1.35 million people worldwide die from road traffic accidents and, in addition, between 20 and 50 million people are injured, placing road traffic accidents as the second leading cause of death among people between the ages of 5 and 29. According to WHO, human errors, such as speeding, driving under the influence of drugs, fatigue, or distractions at the wheel, are the underlying cause of most road accidents. Global reports on road safety such as "Road safety in the European Union. Trends, statistics, and main challenges" prepared by the European Commission in 2018 presented a statistical analysis that related road accident mortality rates and periods segmented by hours and days of the week. This report revealed that the highest incidence of mortality occurs regularly in the afternoons during working days, coinciding with the period when the volume of traffic increases and when any human error is much more likely to cause a traffic accident. Accordingly, mitigating human errors in driving is a challenge, and there is currently a growing trend in the proposal for technological solutions intended to integrate driver information into advanced driving systems to improve driver performance and ergonomics. The study of human factors in the field of driving is a multidisciplinary field in which several areas of knowledge converge, among which stand out psychology, physiology, instrumentation, signal treatment, machine learning, the integration of information and communication technologies (ICTs), and the design of human-machine communication interfaces. The main objective of this thesis is to exploit knowledge related to the different facets of human factors in the field of driving. Specific objectives include identifying tasks related to driving, the detection of unfavorable cognitive states in the driver, such as stress, and, transversely, the proposal for an architecture for the integration and coordination of driver monitoring systems with other active safety systems. It should be noted that the specific objectives address the critical aspects in each of the issues to be addressed. Identifying driving-related tasks is one of the primary aspects of the conceptual framework of driver modeling. Identifying maneuvers that a driver performs requires training beforehand a model with examples of each maneuver to be identified. To this end, a methodology was established to form a data set in which a relationship is established between the handling of the driving controls (steering wheel, pedals, gear lever, and turn indicators) and a series of adequately identified maneuvers. This methodology consisted of designing different driving scenarios in a realistic driving simulator for each type of maneuver, including stop, overtaking, turns, and specific maneuvers such as U-turn and three-point turn. From the perspective of detecting unfavorable cognitive states in the driver, stress can damage cognitive faculties, causing failures in the decision-making process. Physiological signals such as measurements derived from the heart rhythm or the change of electrical properties of the skin are reliable indicators when assessing whether a person is going through an episode of acute stress. However, the detection of stress patterns is still an open problem. Despite advances in sensor design for the non-invasive collection of physiological signals, certain factors prevent reaching models capable of detecting stress patterns in any subject. This thesis addresses two aspects of stress detection: the collection of physiological values during stress elicitation through laboratory techniques such as the Stroop effect and driving tests; and the detection of stress by designing a process flow based on unsupervised learning techniques, delving into the problems associated with the variability of intra- and inter-individual physiological measures that prevent the achievement of generalist models. Finally, in addition to developing models that address the different aspects of monitoring, the orchestration of monitoring systems and active safety systems is a transversal and essential aspect in improving safety, ergonomics, and driving experience. Both from the perspective of integration into test platforms and integration into final systems, the problem of deploying multiple active safety systems lies in the adoption of monolithic models where the system-specific functionality is run in isolation, without considering aspects such as cooperation and interoperability with other safety systems. This thesis addresses the problem of the development of more complex systems where monitoring systems condition the operability of multiple active safety systems. To this end, a mediation architecture is proposed to coordinate the reception and delivery of data flows generated by the various systems involved, including external sensors (lasers, external cameras), cabin sensors (cameras, smartwatches), detection models, deliberative models, delivery systems and machine-human communication interfaces. Ontology-based data modeling plays a crucial role in structuring all this information and consolidating the semantic representation of the driving scene, thus allowing the development of models based on data fusion.I would like to thank the Ministry of Economy and Competitiveness for granting me the predoctoral fellowship BES-2016-078143 corresponding to the project TRA2015-63708-R, which provided me the opportunity of conducting all my Ph. D activities, including completing an international internship.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: José María Armingol Moreno.- Secretario: Felipe Jiménez Alonso.- Vocal: Luis Mart

iMind: Uma ferramenta inteligente para suporte de compreensão de conteúdo

Usually while reading, content comprehension difficulty affects individual performance. Comprehension difficulties, e. g., could lead to a slow learning process, lower work quality, and inefficient decision-making. This thesis introduces an intelligent tool called “iMind” which uses wearable devices (e.g., smartwatches) to evaluate user comprehension difficulties and engagement levels while reading digital content. Comprehension difficulty can occur when there are not enough mental resources available for mental processing. The mental resource for mental processing is the cognitive load (CL). Fluctuations of CL lead to physiological manifestation of the autonomic nervous system (ANS), which can be measured by wearables, like smartwatches. ANS manifestations are, e. g., an increase in heart rate. With low-cost eye trackers, it is possible to correlate content regions to the measurements of ANS manifestation. In this sense, iMind uses a smartwatch and an eye tracker to identify comprehension difficulty at content regions level (where the user is looking). The tool uses machine learning techniques to classify content regions as difficult or non-difficult based on biometric and non-biometric features. The tool classified regions with a 75% accuracy and 80% f-score with Linear regression (LR). With the classified regions, it will be possible, in the future, to create contextual support for the reader in real-time by, e.g., translating the sentences that induced comprehension difficulty.Normalmente durante a leitura, a dificuldade de compreensão pode afetar o desempenho da leitura. A dificuldade de compreensão pode levar a um processo de aprendizagem mais lento, menor qualidade de trabalho ou uma ineficiente tomada de decisão. Esta tese apresenta uma ferramenta inteligente chamada “iMind” que usa dispositivos vestíveis (por exemplo, smartwatches) para avaliar a dificuldade de compreensão do utilizador durante a leitura de conteúdo digital. A dificuldade de compreensão pode ocorrer quando não há recursos mentais disponíveis suficientes para o processamento mental. O recurso usado para o processamento mental é a carga cognitiva (CL). As flutuações de CL levam a manifestações fisiológicas do sistema nervoso autônomo (ANS), manifestações essas, que pode ser medido por dispositivos vestíveis, como smartwatches. As manifestações do ANS são, por exemplo, um aumento da frequência cardíaca. Com eye trackers de baixo custo, é possível correlacionar manifestação do ANS com regiões do texto, por exemplo. Neste sentido, a ferramenta iMind utiliza um smartwatch e um eye tracker para identificar dificuldades de compreensão em regiões de conteúdo (para onde o utilizador está a olhar). Adicionalmente a ferramenta usa técnicas de machine learning para classificar regiões de conteúdo como difíceis ou não difíceis com base em features biométricos e não biométricos. A ferramenta classificou regiões com uma precisão de 75% e f-score de 80% usando regressão linear (LR). Com a classificação das regiões em tempo real, será possível, no futuro, criar suporte contextual para o leitor em tempo real onde, por exemplo, as frases que induzem dificuldade de compreensão são traduzidas

Knowledge management framework based on brain models and human physiology

The life of humans and most living beings depend on sensation and perception for the best assessment of the surrounding world. Sensorial organs acquire a variety of stimuli that are interpreted and integrated in our brain for immediate use or stored in memory for later recall. Among the reasoning aspects, a person has to decide what to do with available information. Emotions are classifiers of collected information, assigning a personal meaning to objects, events and individuals, making part of our own identity. Emotions play a decisive role in cognitive processes as reasoning, decision and memory by assigning relevance to collected information. The access to pervasive computing devices, empowered by the ability to sense and perceive the world, provides new forms of acquiring and integrating information. But prior to data assessment on its usefulness, systems must capture and ensure that data is properly managed for diverse possible goals. Portable and wearable devices are now able to gather and store information, from the environment and from our body, using cloud based services and Internet connections. Systems limitations in handling sensorial data, compared with our sensorial capabilities constitute an identified problem. Another problem is the lack of interoperability between humans and devices, as they do not properly understand human’s emotional states and human needs. Addressing those problems is a motivation for the present research work. The mission hereby assumed is to include sensorial and physiological data into a Framework that will be able to manage collected data towards human cognitive functions, supported by a new data model. By learning from selected human functional and behavioural models and reasoning over collected data, the Framework aims at providing evaluation on a person’s emotional state, for empowering human centric applications, along with the capability of storing episodic information on a person’s life with physiologic indicators on emotional states to be used by new generation applications

Investigation of an emotional virtual human modelling method

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.In order to simulate virtual humans more realistically and enable them life-like behaviours, several exploration research on emotion calculation, synthetic perception, and decision making process have been discussed. A series of sub-modules have been designed and simulation results have been presented with discussion. A visual based synthetic perception system has been proposed in this thesis, which allows virtual humans to detect the surrounding virtual environment through a collision-based synthetic vision system. It enables autonomous virtual humans to change their emotion states according to stimuli in real time. The synthetic perception system also allows virtual humans to remember limited information within their own First-in-first-out short-term virtual memory. The new emotion generation method includes a novel hierarchical emotion structure and a group of emotion calculation equations, which enables virtual humans to perform emotionally in real-time according to their internal and external factors. Emotion calculation equations used in this research were derived from psychologic emotion measurements. Virtual humans can utilise the information in virtual memory and emotion calculation equations to generate their own numerical emotion states within the hierarchical emotion structure. Those emotion states are important internal references for virtual humans to adopt appropriate behaviours and also key cues for their decision making. The work introduces a dynamic emotional motion database structure for virtual human modelling. When developing realistic virtual human behaviours, lots of subjects were motion-captured whilst performing emotional motions with or without intent. The captured motions were endowed to virtual characters and implemented in different virtual scenarios to help evoke and verify design ideas, possible consequences of simulation (such as fire evacuation). This work also introduced simple heuristics theory into decision making process in order to make the virtual human’s decision making more like real human. Emotion values are proposed as a group of the key cues for decision making under the simple heuristic structures. A data interface which connects the emotion calculation and the decision making structure together has also been designed for the simulation system

Does insecure attachment lead to (mis)wired brains? Emotion, cognition, and attachment: an outlook through psychophysiological pathways

2346, 2360, 2560The evolutionary-based attachment theory (Bowlby, 1969, 1973, 1980) asserts that approach/attachment or avoidance/withdrawal tendencies may reflect distinct regulation strategies underlying individual differences in attachment styles. The influence of the internal working models of attachment on emotion and cognition, and more recently, on its psychophysiological underpinnings has been a central focus of research. Despite the endeavours at clarifying this modulatory influence in behaviour, inconsistent results have prevented definite answers. Aiming at contributing to the current knowledge in the filed, and embedded in a psychophysiological framework, the present thesis brings together findings of empirical studies focusing on the regulation abilities in attentional bias towards emotion information. Following an integrative approach, these studies coupled behavioural responses with measures of skin conductance, heart rate, and eye movements. Findings of these studies converge to show distinctive features between regulation strategies deployed by insecure attached individuals when processing threat-related information on visual attention tasks, as measured by behavioural (Study I), sympathetic (Study II), and eye movement (Study III) responses. Taken together these findings point up the evolutionary value of the attachment behavioural system, providing support for fundamental distinctions between insecure attachment styles, both at a behavioural and physiological level. Considering recent advances emerging in the filed, results are discussed within in a comprehensive and all-encompassing approach.Fundamentada num cenário evolucionista, a teoria da vinculação (Bowlby, 1969, 1973, 1980) considera que comportamentos de aproximação/evitamento reflectem estratégias de regulação subjacentes a diferenças individuais nos estilos de vinculação. Neste âmbito, a natureza dos modelos internos dinâmicos têm sido um foco central na investigação, tendo sido dada particular atenção à sua influência nos processos emocionais e cognitivos e, mais recentemente, às suas bases psicofisiológicas. Contudo, apesar de vários estudos terem examinado estas questões, a ausência de dados consistentes acerca dos mecanismos que poderão contribuir para esta influência estão ainda por conhecer de modo consistente. Visando contribuir para o conhecimento neste campo, a presente tese reúne um conjunto de estudos empíricos que, numa perspectiva psicofisiológica, focam a acção das estratégias de regulação associadas aos estilos de vinculação insegura – ansiosa e evitante –, nos enviesamentos atencionais no processamento de informação emocional. Numa abordagem integrativa, estes estudos combinam respostas comportamentais com medidas fisiológicas: condutância da pele; frequência cardíaca; e movimentos oculares. Utilizando tarefas de atenção visual, os resultados destes estudos apoiam a hipótese de que os estilos de vinculação insegura estão relacionados com estratégias de regulação específicas no processamento de estímulos potencialmente ameaçadores, avaliadas através de respostas comportamentais (Estudo I), do sistema nervoso simpático (Estudo II), e dos movimentos oculares (Estudo III). Globalmente, os resultados corroboraram o valor evolutivo do sistema comportamental de vinculação, dando suporte para diferenças entre os estilos de vinculação insegura, tanto a nível comportamental como fisiológico. Considerando progressos científicos emergentes, os resultados são discutidos numa abordagem compreensiva e abrangente

From Photography to fMRI

Hysteria, a mysterious disease known since antiquity, is said to have ceased to exist. Challenging this commonly held view, this is the first cross-disciplinary study to examine the current functional neuroimaging research into hysteria and compare it to the nineteenth-century image-based research into the same disorder. Paula Muhr's central argument is that, both in the nineteenth-century and the current neurobiological research on hysteria, images have enabled researchers to generate new medical insights. Through detailed case studies, Muhr traces how different images, from photography to functional brain scans, have reshaped the historically situated medical understanding of this disorder that defies the mind-body dualism

From Photography to fMRI

Hysteria, a mysterious disease known since antiquity, is said to have ceased to exist. Challenging this commonly held view, this is the first cross-disciplinary study to examine the current functional neuroimaging research into hysteria and compare it to the nineteenth-century image-based research into the same disorder. Paula Muhr's central argument is that, both in the nineteenth-century and the current neurobiological research on hysteria, images have enabled researchers to generate new medical insights. Through detailed case studies, Muhr traces how different images, from photography to functional brain scans, have reshaped the historically situated medical understanding of this disorder that defies the mind-body dualism

From Photography to fMRI: Epistemic Functions of Images in Medical Research on Hysteria

Hysteria, a mysterious disease known since antiquity, is said to have ceased to exist. Challenging this commonly held view, this is the first cross-disciplinary study to examine the current functional neuroimaging research into hysteria and compare it to the nineteenth-century image-based research into the same disorder. Paula Muhr's central argument is that, both in the nineteenth-century and the current neurobiological research on hysteria, images have enabled researchers to generate new medical insights. Through detailed case studies, Muhr traces how different images, from photography to functional brain scans, have reshaped the historically situated medical understanding of this disorder that defies the mind-body dualism