Test and Evaluation Metrics of Crew Decision-Making And Aircraft Attitude and Energy State Awareness

NASA has established a technical challenge, under the Aviation Safety Program, Vehicle Systems Safety Technologies project, to improve crew decision-making and response in complex situations. The specific objective of this challenge is to develop data and technologies which may increase a pilot's (crew's) ability to avoid, detect, and recover from adverse events that could otherwise result in accidents/incidents. Within this technical challenge, a cooperative industry-government research program has been established to develop innovative flight deck-based counter-measures that can improve the crew's ability to avoid, detect, mitigate, and recover from unsafe loss-of-aircraft state awareness - specifically, the loss of attitude awareness (i.e., Spatial Disorientation, SD) or the loss-of-energy state awareness (LESA). A critical component of this research is to develop specific and quantifiable metrics which identify decision-making and the decision-making influences during simulation and flight testing. This paper reviews existing metrics and methods for SD testing and criteria for establishing visual dominance. The development of Crew State Monitoring technologies - eye tracking and other psychophysiological - are also discussed as well as emerging new metrics for identifying channelized attention and excessive pilot workload, both of which have been shown to contribute to SD/LESA accidents or incidents

An Electroencephalographic Investigation of the Encoding of Sound Source Elevation in the Human Cortex

Sound localization is of great ecological importance because it provides spa- tial perception outside the visual field. However, unlike other sensory systems, the auditory system does not represent the location of a stimulus on the level of the sensory epithelium in the cochlea. Instead, the position of a sound source has to be computed based on different localization cues. Different cues are informative of a sound sources azimuth and elevation, which, when taken together, describe the sources location in a polar coordinate system. There is a body of knowledge regarding the acoustical cues and the neural circuits in the brainstem required to perceive sound source azimuth and elevation. However, our understanding of the encoding of sound source location on the level of the cortex is lacking especially what concerns elevation. Within the scope of this thesis, we established an experimental setup to study auditory spatial perception while recording the listeners brain activity using electroencephalography. We conducted two experiments on the encoding of sound source elevation in the human cortex. Both experiments results are compatible with the hypothesis that the cortex represents sound source elevation in a population rate code where the response amplitude decreases linearly with increasing elevation. Decoding of the recorded brain activity revealed that a distinct neural representation of differently elevated sound sources was predictive of behavioral performance. An exploratory analysis indicated an increase in the amplitude of oscillations in visual areas when the subject localized sounds during eccentric eye positions. More research in this direction could help shed light on the interactions between the visual and auditory systems regarding spatial perception. The experiments presented in this dissertation are, to our knowledge, the first studies that demonstrate the encoding of sound source elevation in the human cortex by using a direct measure of neural activity (i.e., electroencephalography).:Abstract . . . . . . . . . . . . . . . . . . . . . . 1 Zusammenfassung . . . . . . . . . . . . . . . . . . . . . . 7 1 Electroencephalography 13 1.1 Event Related Potentials and Oscillations . . . . . . . . . . . . 13 1.2 Comparison to other Methods . . . . . . . . . . . . . . . . . . . 14 1.3 EEG Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4.1 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4.2 Referencing . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4.3 Eye Blinks . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.4.4 Epoch Rejection . . . . . . . . . . . . . . . . . . . . . . . 22 1.4.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.5 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.5.1 Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.5.2 Nonparametric Permutation Testing . . . . . . . . . . . 26 1.5.3 Source Separation . . . . . . . . . . . . . . . . . . . . . . 28 2 Sound Localization in the Brain . . . . . . . . . . . . . . . . . . . . 31 2.1 The Spatial Perception of Sound . . . . . . . . . . . . . . . . . . 32 2.1.1 Interaural Cues . . . . . . . . . . . . . . . . . . . . . . . 32 2.1.2 Spectral Cues . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2 Brain Mechanisms for Sound Localization . . . . . . . . . . . . 37 2.2.1 Auditory Pathway . . . . . . . . . . . . . . . . . . . . . 38 2.2.2 Extracting Localization Cues . . . . . . . . . . . . . . . 40 2.2.3 Neural Representation of Auditory Space . . . . . . . . 42 2.2.4 The Dual Pathway Model . . . . . . . . . . . . . . . . . 45 2.2.5 A Dominant Hemisphere for Sound Localization? . . . 47 2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3 A Free Field Setup for Psychoacoustics 51 3.1 Design of the Experimental Setup . . . . . . . . . . . . . . . . . 51 3.1.1 Loudspeakers . . . . . . . . . . . . . . . . . . . . . . . . 54 3.1.2 Processors . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.1.3 Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.1.4 Coordinate Systems . . . . . . . . . . . . . . . . . . . . 56 3.2 Operating the Setup . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.2.1 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.2.2 Loudspeaker Equalization . . . . . . . . . . . . . . . . . 59 3.3 Head Pose Estimation . . . . . . . . . . . . . . . . . . . . . . . 61 3.3.1 Landmark Detection . . . . . . . . . . . . . . . . . . . . 62 3.3.2 Perspective-n-Point Problem . . . . . . . . . . . . . . . 62 3.3.3 Camera-to-World Conversion . . . . . . . . . . . . . . . 63 3.4 A Toolbox for Psychoacoustics . . . . . . . . . . . . . . . . . . 64 4 A Linear Population Rate Code for Elevation 67 4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.1.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.1.2 Experimental Protocol . . . . . . . . . . . . . . . . . . . 69 4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.2.1 Behavioral Performance . . . . . . . . . . . . . . . . . . 70 4.2.2 ERP Components . . . . . . . . . . . . . . . . . . . . . . 70 4.2.3 Elevation Encoding . . . . . . . . . . . . . . . . . . . . . 72 4.2.4 Effect of Eye-Position . . . . . . . . . . . . . . . . . . . . 74 4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5 Decoding of Brain Responses Predicts Localization Accuracy . . . 81 5.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.1.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.1.2 Experimental Protocol . . . . . . . . . . . . . . . . . . . 82 5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.2.1 Behavioral Performance . . . . . . . . . . . . . . . . . . 83 5.2.2 ERP Components . . . . . . . . . . . . . . . . . . . . . . 84 5.2.3 Decoding Brain Activity . . . . . . . . . . . . . . . . . . 86 5.2.4 Topography of Elevation Encoding . . . . . . . . . . . . 88 5.2.5 Elevation Tuning . . . . . . . . . . . . . . . . . . . . . . 89 5.2.6 Hemispheric Lateralization . . . . . . . . . . . . . . . . 91 5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 A Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 B Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Research on Application of Cognitive-Driven Human-Computer Interaction

Human-computer interaction is an important research content of intelligent manufacturing human factor engineering. Natural human-computer interaction conforms to the cognition of users' habits and can efficiently process inaccurate information interaction, thus improving user experience and reducing cognitive load. Through the analysis of the information interaction process, user interaction experience cognition and human-computer interaction principles in the human-computer interaction system, a cognitive-driven human-computer interaction information transmission model is established. Investigate the main interaction modes in the current human-computer interaction system, and discuss its application status, technical requirements and problems. This paper discusses the analysis and evaluation methods of interaction modes in human-computer system from three levels of subjective evaluation, physiological measurement and mathematical method evaluation, so as to promote the understanding of inaccurate information to achieve the effect of interaction self-adaptation and guide the design and optimization of human-computer interaction system. According to the development status of human-computer interaction in intelligent environment, the research hotspots, problems and development trends of human-computer interaction are put forward

Human Machine Interfaces for Teleoperators and Virtual Environments

In Mar. 1990, a meeting organized around the general theme of teleoperation research into virtual environment display technology was conducted. This is a collection of conference-related fragments that will give a glimpse of the potential of the following fields and how they interplay: sensorimotor performance; human-machine interfaces; teleoperation; virtual environments; performance measurement and evaluation methods; and design principles and predictive models

Multi-modal virtual environment research at Armstrong Laboratory

One mission of the Paul M. Fitts Human Engineering Division of Armstrong Laboratory is to improve the user interface for complex systems through user-centered exploratory development and research activities. In support of this goal, many current projects attempt to advance and exploit user-interface concepts made possible by virtual reality (VR) technologies. Virtual environments may be used as a general purpose interface medium, an alternative display/control method, a data visualization and analysis tool, or a graphically based performance assessment tool. An overview is given of research projects within the division on prototype interface hardware/software development, integrated interface concept development, interface design and evaluation tool development, and user and mission performance evaluation tool development

Proceedings of the EAA Spatial Audio Signal Processing symposium: SASP 2019

International audienc

Corticothalamic feedback sculpts visual spatial integration in mouse thalamus

En route from retina to cortex, visual information travels through the dorsolateral geniculate nucleus of the thalamus (dLGN), where extensive cortico-thalamic (CT) feedback has been suggested to modulate spatial processing. How this modulation arises from direct excitatory and indirect inhibitory CT feedback components remains enigmatic. We show that in awake mice topographically organized cortical feedback modulates spatial integration in dLGN by sharpening receptive fields (RFs) and increasing surround suppression. Guided by a network model revealing wide-scale inhibitory CT feedback necessary to reproduce these effects, we targeted the visual sector of the thalamic reticular nucleus (visTRN) for recordings. We found that visTRN neurons have large receptive fields, show little surround suppression, and have strong feedback-dependent responses to large stimuli, making them an ideal candidate for mediating feedback-enhanced surround suppression in dLGN. We conclude that cortical feedback sculpts spatial integration in dLGN, likely via recruitment of neurons in visTRN

Methods of visualization and analysis of cardiac depolarization in the three dimensional space

The master thesis presents methods for intellectual analysis and visualization 3D EKG in order to increase the efficiency of ECG analysis by extracting additional data. Visualization is presented as part of the signal analysis tasks considered imaging techniques and their mathematical description. Have been developed algorithms for calculating and visualizing the signal attributes are described using mathematical methods and tools for mining signal. The model of patterns searching for comparison purposes of accuracy of methods was constructed, problems of a clustering and classification of data are solved, the program of visualization of data is also developed. This approach gives the largest accuracy in a task of the intellectual analysis that is confirmed in this work. Considered visualization and analysis techniques are also applicable to the multi-dimensional signals of a different kind