Predicting Inattentional Blindness with Pupillary Response in a Simulated Flight Task

Inattentional blindness (IB) is the failure of observers to notice the presence of a clearly viewable but unexpected visual event when attentional resources are diverted elsewhere. Knowing when an operator is unable to respond or detect an unexpected event may help improve safety during task performance. Unfortunately, it is difficult to predict when such failures might occur. The current study was a secondary data analysis of data collected in the Human and Autonomous Vehicle Systems Laboratory at NASA Langley Research Center. Specifically, 60 subjects (29 male, with normal or corrected-to-normal vision, mean age of 34.5 years (SD = 13.3) were randomly assigned to one of three automation conditions (full automation, partial automation, and full manual) and took part in a simulated flight landing task. The dependent variable was the detection/non-detection of an IB occurrence (a truck on the landing runway). Scores on the NASA-TLX workload rating scale varied significantly by automation condition. The full automation condition reported the lowest subjective task load followed by partial automation and then manual condition. IB detection varied significantly across automation condition. The moderate workload condition of partial automation exhibited the lowest likelihood of IB occurrence. The low workload full automation condition did not differ significantly from the manual condition. Subjects who reported higher task demand had increased pupil dilation and subjects with larger pupil dilation were more likely to detect the runway incursion. These results show eye tracking may be used to identify periods of reduced unexpected visual stimulus detection for possible real-time IB mitigation

Analysis and synthesis of iris images

Of all the physiological traits of the human body that help in personal identification, the iris is probably the most robust and accurate. Although numerous iris recognition algorithms have been proposed, the underlying processes that define the texture of irises have not been extensively studied. In this thesis, multiple pair-wise pixel interactions have been used to describe the textural content of the iris image thereby resulting in a Markov Random Field (MRF) model for the iris image. This information is expected to be useful for the development of user-specific models for iris images, i.e. the matcher could be tuned to accommodate the characteristics of each user\u27s iris image in order to improve matching performance. We also use MRF modeling to construct synthetic irises based on iris primitive extracted from real iris images. The synthesis procedure is deterministic and avoids the sampling of a probability distribution making it computationally simple. We demonstrate that iris textures in general are significantly different from other irregular textural patterns. Clustering experiments indicate that the synthetic irises generated using the proposed technique are similar in textural content to real iris images

Exploring cognition in visual search and vigilance tasks with eye tracking and pupillometry

Recent findings in experimental psychology suggest that pupillometry, the measurement of pupil size, can provide insight into cognitive processes associated with effort and target detection in visual search tasks and monitoring performance in vigilance tasks. With the increasing availability, affordability and flexibility of video-based eye tracking hardware, these experimental findings point to lucrative practical applications such as real-time biobehavioural monitoring systems to assist with socially important tasks in operational settings. The aim of the current thesis was to explore this potential with further experimental work paying close attention to methodological issues which complicate cognitive interpretations of pupillary responses, such as physical stimulus confounds and eye movement-related measurement error in video-based systems. Six original experiments were designed to specifically explore the relationship between pupil size, cognition and behavioural performance in classic visual search and vigilance paradigms. Experiments 1-2 examined the pupillometric effects of effort and target detection in visual search with briefly presented stimuli. Pupil responses showed small variability with respect to manipulations of set size and target presence but were influenced substantially by the requirement for a motor response. Experiments 3-4 explored the cognitive pupil dynamics of free-viewing visual search with data-driven correction for eye movement artefacts. Group-level averages revealed small transient pupil dilations following fixations on targets but not distractors, an effect which was not contingent on a motor response or correction for gaze position artefacts. Experiments 5-6 looked at the relationship between pupil size and detection performance measures in two types of vigilance task. Changes in baseline and stimulus-evoked pupil responses loosely mirrored changes in performance, but the relationships were neither linear nor consistent. Overall, the thesis affirms the practical potential for using cognitive pupillometry in research and applied settings, but emphasises the constraints arising from methodological and theoretical limitations

Les yeux ont des oreilles : examen de la pupillométrie en tant qu'indice psychophysiologique de la capture attentionnelle auditive

Tableau d'honneur de la Faculté des études supérieures et postdoctorales, 2018-2019.La présence de sons non pertinents est reconnue pour affecter le fonctionnement cognitif. Plus précisément, la présentation d’un son qui dévie du contexte auditif a le potentiel d’affecter la performance à une tâche réalisée simultanément. Plusieurs études s’intéressant à l’impact délétère des sons déviants sur la performance à une tâche montrent qu’il découle de la réorientation de l’attention déclenchée lorsqu’un son déviant est détecté par l’organisme. Cette capture attentionnelle (ou réponse d’orientation) est également reconnue pour engendrer plusieurs réponses physiologiques associées à l’état d’alerte. Ces réponses physiologiques, sous certaines conditions, sont considérées comme des indices psychophysiologiques de la capture attentionnelle auditive permettant de démontrer l’occurrence d’une réorientation de l’attention de la tâche en cours vers le son déviant. De récentes études suggèrent qu’il existe un lien entre ces indices et la réponse pupillaire, soit l’augmentation rapide du diamètre pupillaire. Quelques études ont tenté d’évaluer si cette réponse pouvait remplir les critères d’un indice valide de la réponse d’orientation; cependant, les résultats de ces études sont parfois contradictoires ou incomplets. La présente thèse propose donc une évaluation systématique de l’utilisation de la réponse pupillaire à titre d’indice psychophysiologique de la capture attentionnelle auditive. Les résultats de la première étude montrent que la réponse pupillaire respecte les critères d’un index valide de la réponse d’orientation, supportant ainsi la validité de cet index. L’Étude 2 montre que cet indice peut être utilisé dans des contextes dans lesquels les sujets effectuent simultanément à l’écoute des sons une tâche visuelle provoquant des changements systématiques de luminosité ou des mouvements oculaires qui affectent la taille de la pupille. Ces expériences appuient ainsi l’utilisabilité de cet index. Enfin, la dernière étude permet d’établir l’utilité de ce proxy puisque ce dernier permet de distinguer l’origine de deux phénomènes de distraction auditive différents. Dans l’ensemble, les résultats de la thèse révèlent que la réponse pupillaire représente un indice psychophysiologique adéquat qui pourrait être intégré aux études sur la distraction auditive ou dans des contextes plus appliqués où les mesures de la capture attentionnelle auditive peuvent s’avérer pertinentes.The presence of task-irrelevant sound is known to impede cognitive functioning. More precisely, presenting a sound that deviates from the auditory background has been shown to disrupt performance on an ongoing task. Several studies regarding performance disruption by deviant sounds showed that this effect originates from the reorientation of attention triggered when the organism has detected the deviant sound. Such attentional capture (or orientation response) also elicits many physiological responses related to one’s state of alertness. These physiological responses, under certain conditions, are considered as psychophysiological indices of auditory attentional capture, permitting to show that attention has been reoriented from an ongoing task toward the deviant auditory stimulus. Recent work suggested a relationship between these indices and the pupillary dilation response, that is a rapid increase in the pupil size. A few studies have attempted to assess whether the pupillary dilation response could respect the criteria for an index to be considered as a proxy for attentional capture; yet, results of these studies either lack consistency or are incomplete. Hence, the current thesis proposes a systematic assessment of using the pupillary dilation response as a psychophysiological proxy for the auditory attentional capture. Results of the first study suggest that the pupillary dilation response respects criteria of a valid auditory attentional capture index, thus supporting the validity of this index. Study 2 shows that this index can be used in contexts in which participants are exposed to irrelevant deviant sound while concurrently performing a task inducing luminance changes or ocular movements, both influencing the pupil size. These experiments hence supported the usability of this index. The last study allows establishing the utility of this proxy since it can be used to dissociate the origin of two different auditory distraction phenomena. Overall, results of the current thesis reveal that the pupillary dilation response is a psychophysiological index that could be added to several auditory distraction paradigms or in applied contexts in which the occurrence of attention reorientation toward a sound may be relevant to assess

Human-centred design for next generation of air traffic management systems.

Designing and deploying air traffic management systems requires an understanding of cognitive ergonomics, system integration, and human-computer interactions. The aim of this research is to develop an effective Human-centred design for Air Navigation Services Providers to permit more effective air traffic controller training and regulations. Therefore, this research consists of both evaluating human-computer interactions on COOPANS Air Traffic Management system and multiple remote tower operations. The COOPANS Alliance is an international cooperation among the air navigation service providers of Austria, Croatia, Denmark, Ireland, Portugal and Sweden with Thales as the industry supplier. The findings of this project indicate that the context-specified design of semantic alerts could improve ATCO’s situational awareness and significantly reduce response time when responding to aircraft conflict resolution alerts. Civil Aviation Authorities, Air Navigation Service Providers and Air Traffic Management System Providers could all benefit from the findings of this research with a view to ensuring that Air Traffic Controllers are provided with the optimal context-specified alerting schemes to increase their situational awareness during both training and operations. The EU Single European Sky initiative was introduced to restructure European airspace and propose innovative measures for air traffic management to achieve the objectives of enhanced cost-efficiency and improved airspace design and airport capacity whilst simultaneously improving safety performance. There is potential to save approximately €2.21 million Euro per annum per installation of remote tower versus traditional control towers. However, ATCO’s visual attention and monitoring performance can be affected by how information is presented, the complexity of the information presented, and the operating environment in the remote tower centre. To achieve resource-efficient and sustainable air navigation services, there is a need to improve the design of human-computer interactions in multiple remote tower technology deployment. These must align with high technology-readiness levels, operators’ practices, industrial developments, and the certification processes of regulators. From a regulatory perspective the results of this project may contribute to European Aviation Safety Agency rulemaking activity for future Air Traffic Management Systems. Overall, the results of this research are in line with the requirements of Single European Sky and facilitate the harmonisation of European ATM systems.PhD in Transport System

Deep learning for diabetic retinopathy detection and classification based on fundus images: A review.

Diabetic Retinopathy is a retina disease caused by diabetes mellitus and it is the leading cause of blindness globally. Early detection and treatment are necessary in order to delay or avoid vision deterioration and vision loss. To that end, many artificial-intelligence-powered methods have been proposed by the research community for the detection and classification of diabetic retinopathy on fundus retina images. This review article provides a thorough analysis of the use of deep learning methods at the various steps of the diabetic retinopathy detection pipeline based on fundus images. We discuss several aspects of that pipeline, ranging from the datasets that are widely used by the research community, the preprocessing techniques employed and how these accelerate and improve the models' performance, to the development of such deep learning models for the diagnosis and grading of the disease as well as the localization of the disease's lesions. We also discuss certain models that have been applied in real clinical settings. Finally, we conclude with some important insights and provide future research directions

Human-Centric Machine Vision

Recently, the algorithms for the processing of the visual information have greatly evolved, providing efficient and effective solutions to cope with the variability and the complexity of real-world environments. These achievements yield to the development of Machine Vision systems that overcome the typical industrial applications, where the environments are controlled and the tasks are very specific, towards the use of innovative solutions to face with everyday needs of people. The Human-Centric Machine Vision can help to solve the problems raised by the needs of our society, e.g. security and safety, health care, medical imaging, and human machine interface. In such applications it is necessary to handle changing, unpredictable and complex situations, and to take care of the presence of humans

Determinants of paediatric asthma: a three-level approach

Tese de doutoramento em Ciências Biomédicas, apresentada ao Instituto de Ciências Biomédicas Abel Salazar da Universidade do Porto, 2021Orientador Professor André Moreira (Faculdade de Medicina da Universidade do Porto); Coorientadores: Professor Eduardo de Oliveira Fernandes (Faculdade de Engenharia da Universidade do Porto); Doutora Joana Madureira (Instituto de Saúde Pública da Universidade do Porto); Professora Idalina Beirão (Instituto de Ciências Biomédicas Abel Salazar da Universidade do Porto)During the 20th century, urbanization has increasing and represented a major demographic and environmental change in developed countries. Urban living may offer a greater possibility to better health care, education and social services, but is also associated with increased exposure to air pollution, outdoors and indoors, loss of natural environments and biodiversity and lifestyle changes. Furthermore, this ever-changing urban environment has an impact on diseases patterns and prevalence, namely on noncommunicable diseases, such as asthma and allergy, and poses many challenges to understand the relationship between the changing on the urban environment and the children health. The overall aim of this thesis was to study the role of the environmental determinants of paediatric asthma.Ao longo do último século observou-se uma tendência crescente e rápida na urbanização, representando uma grande mudança demográfica e ambiental, principalmente nos países desenvolvidos. Apesar das cidades poderem oferecer oportunidades de acesso aos serviços de saúde, educação e sociais, estão também associadas a um aumento da exposição à poluição do ar, no exterior e no interior, diminuição de ambientes naturais e da biodiversidade e alterações nos estilos de vida. A alteração crescente do meio ambiente está igualmente associada ao aumento da prevalência de algumas doenças, nomeadamente, de doenças crónicas como a asma e as alergias e, representando por isso diversos desafios na compreensão da relação entre as mudanças no ambiente na saúde das crianças. O objetivo da presente tese é investigar o papel dos determinantes ambientais da asma e da alergia pediátrica.This work was supported by Fundação para a Ciência e Tecnologia through the scholarship SFRH/BD/112269/2015 and by the Project NORTE-01-0145-FEDER-000010 – Health, Comfort and Energy in the Built Environment (HEBE), cofinanced by Programa Operacional Regional do Norte (NORTE2020), through Fundo Europeu de Desenvolvimento Regional (FEDER), EXALAR 21 project financed by FEDER/FNR and by Fundação para a Ciência e Tecnologia (EXALAR 21 02/SAICT/2017 - Project nº 30193) and by EAACI Research Fellowship 2018, “How indoor environment can shape human microbiome: culture-independent approaches”info:eu-repo/semantics/publishedVersio

On the Recognition of Emotion from Physiological Data

This work encompasses several objectives, but is primarily concerned with an experiment where 33 participants were shown 32 slides in order to create ‗weakly induced emotions‘. Recordings of the participants‘ physiological state were taken as well as a self report of their emotional state. We then used an assortment of classifiers to predict emotional state from the recorded physiological signals, a process known as Physiological Pattern Recognition (PPR). We investigated techniques for recording, processing and extracting features from six different physiological signals: Electrocardiogram (ECG), Blood Volume Pulse (BVP), Galvanic Skin Response (GSR), Electromyography (EMG), for the corrugator muscle, skin temperature for the finger and respiratory rate. Improvements to the state of PPR emotion detection were made by allowing for 9 different weakly induced emotional states to be detected at nearly 65% accuracy. This is an improvement in the number of states readily detectable. The work presents many investigations into numerical feature extraction from physiological signals and has a chapter dedicated to collating and trialing facial electromyography techniques. There is also a hardware device we created to collect participant self reported emotional states which showed several improvements to experimental procedure