This paper describes the application of an important tool which can capture shifting information from pilots' visual attention data. In order to investigate the shifting information, the shifting state space is defined by visual tracking, visual interference and visual dormancy. Using this analysis, the movement of pilots' visual attention can be completely measured. The results that the forecast of the probability shifting model is coincident with the fact suggest the use of the model as a powerful technique for measuring the movement of pilots' visual attention. Furthermore, the link between visual attention and driving experience or sexual distinction are also discussed in the probability shifting model

Heng XIANG

Jie FAN

Keqiang DONG

Directory of Open Access Journals

Sensors & Transducers, Vol. 158, Issue 11, November 2013, pp. 357-362  357  Sensors & Transducers © 2013 by IFSAhttp://www.sensorsportal.com       The Movement Model of Pilots’ Visual Attention  1 Heng XIANG, 2 Jie FAN, 3 Keqiang DONG 1 Sino-European Institute of Aviation Engineering, Civil Aviation University of China,  Tianjin 300300, China 2 College of Science, Hebei University of Engineering,  Handan 056038, China 3 College of Science, Civil Aviation University of China,  Tianjin 300300, China 1 E-mail: hongzhangdong@163.com   Received: 25 August 2013   /Accepted: 25 October 2013   /Published: 30 November 2013   Abstract: This paper describes the application of an important tool which can capture shifting information from pilots’ visual attention data. In order to investigate the shifting information, the shifting state space is defined by visual tracking, visual interference and visual dormancy. Using this analysis, the movement of pilots’ visual attention can be completely measured. The results that the forecast of the probability shifting model is coincident with the fact suggest the use of the model as a powerful technique for measuring the movement of pilots’ visual attention. Furthermore, the link between visual attention and driving experience or sexual distinction are also discussed in the probability shifting model. Copyright © 2013 IFSA  Keywords: Shifting state space, Probability shifting model, Visual attention, Movement, Pilots.    1. Introduction  This paper addresses the task of providing the ground work for a logically formulated and computationally motivated model of attention in visual perception. Attention that moves was first experimentally investigated by Helmholtz who found that one can read a letter in periphery presented briefly enough to prevent eye movements if he attended the location of the letter. However, intensive research such as movement models and algorithms has been generated by the research of machine vision over the last several decades on account of their importance to the machine vision application including surveillance, driver assistance and artificial robot system. Although a lot of outstanding model and modified algorithms are proposed [1, 2], most of them are dependent on special assumption and some of them is not adaptable to difficult and changeable environments in different video sequences. In spite of this, previous work addressed several aspects of human visual attention, including generation of autonomic scan paths of eye gaze movements, aspects of secondary and covert attention, and of exogenous and endogenous attention. Moreover, Scholl provides a detailed discussion of the issues involved, highlighting the difficulties in defining what should constitute an object relative to attention and raising the question as to whether notions of objects are formed pre-attentively or post-attentively [3]. In part visual attention is task or activity dependent, controlled by higher level cognitive processes that select what is to be viewed preferentially according to the current role, for instance, as indicated by visual search tasks [4, 5]. Article number P_1511Sensors & Transducers, Vol. 158, Issue 11, November 2013, pp. 357-362  358According to the findings in psychology, selective visual attention acts like a filter to select the useful information out of the deluge information of the image what is the nature’s answer to computational difficult and changeable environments [6]. On the early selection process, human visual attention is attracted by the salient objects based on innate principles obtained through evolution and shifts between them. And on the late selection process, human brain may adopt a need-based approach that uses task specification and learning knowledge to detect the desired object [7]. Some phenomenon has always been to provide explanations for the characteristics of biological. Reynolds et al proposed a quantification of the biased competition model [8, 9]. Subsequently, Lee and Maunsell appeared a normalization model, the normalization model of attentional modulation, showing how attention changes the gain of responses to individual stimuli and why attentional modulation is more than a gain change when multiple stimuli are present in a receptive field [10].  However, a model of visual attention shifting addresses the observed or predicted behavior of human and non-human primate visual attention. Models, from mathematical, psychological to biological, explain or predict some or all of visual attentive behavior [11-18]. It is necessary to measure the visual tracking, visual interference, and visual dormancy for understanding the movement  of visual attention. Consequently, this paper presents a novel probability shifting model to measure the movement of pilots’ visual attention for dealing with these troubles. In probability shifting model, the shifting state space is defining by state vector such as visual tracking, visual interference and visual dormancy. The probability of three state vectors are all investigated to measure the movement  of visual attention. The rest of the paper is organized as follows. Section 2 gives the overview and derivation of the probability shifting model. Essential experiment and analysis of probability shifting model are detailed in section 3. Section 4 presents the important conclusion of this paper.   2. Probability Shifting Model  Measuring the movements of attention is not as easy as measuring eye movements and the proposed probability shifting model tries to use the three essential processes of selective attention to gain more information about the direction of estimation when target disappears in the scene. A brief description of the proposed probability shifting model algorithm is given in this section. For convenience, the shifting state space of pilots’ visual attention is defining by visual tracking, visual interference and visual dormancy, where visual tracking show that driving information is disposing, visual interference indicate that disposing of driving information is void by unrelated information., visual dormancy mean that disposing of driving information is void owing to physical and psychological trouble. In order to make derivation convenient, we set the state space denoted by:  },,{ CBA=Ω ,  where Ω is the whole attention event, A  is the event of visual tracking, B  is the event of visual interference, and C  is the event of visual dormancy.   2.1. Shifting State Probability  Previous research had found that the movement of visual attention follows the exponential  behavior [19, 20]:   tiietG λ−−=1)( ,   where 1iitλ = , it is the mean of duration at state vector i , ,i A B= , and C . Therefore, the shifting probability between A  and A  is derived as follows:  })()({)(, AtXAttXPtP AA ==∆+=∆   }{ tTP A ∆>=   )(1 tGA−=   )(1 ttA ∆+∆−= ολ   where the random variable ( )X t denote the state vector of visual attention at i , ,i A B= , and C . The shifting probability between A  and B  is expressed as )(, tP BA ∆  and shifting probability between A  and C  is expressed as )(, tP CA ∆ . Based on the maximum likelihood estimator (MLE), the shifting probability from A  to B  or C  is actually in proportion to the mean of duration at state vector i , Bi = , and C . On the other hand, the shifting probability satisfies following conditions  )(1)()( ,,, tPtPtP AACABA ∆−=∆+∆   Therefore,   )()(, tttPCBCBA ∆+∆+=∆ ολλλ  Sensors & Transducers, Vol. 158, Issue 11, November 2013, pp. 357-362  359)()(, tttPCBBCA ∆+∆+=∆ ολλλ   The shifting probability between A , B and C are roughly analogous:   )()()()()(1)()()()()()(1)()()()()()(1)(,,,,,,,,,tttPtttPtttPtttPtttPtttPtttPtttPtttPBAABCBABACCCCCAACBCACABBBBCBBCACBCBAAAA∆+∆+=∆∆+∆+=∆∆+∆−=∆∆+∆+=∆∆+∆+=∆∆+∆−=∆∆+∆+=∆∆+∆+=∆∆+∆−=∆ολλλολλλολολλλολλλολολλλολλλολ, (1)  Further, the shifting matrix can be calculated.   2.2. Model and Problem Solving  Based on the theory of Markov process, the differential equation is obtained by Eq. 1:   [ ])(),(),( tptptp CBA ′′′  [ ]Htptptp CBA )(),(),(= , (2)  where H  is the shifting state matrix obtained from section 2.1. This probability satisfies following conditions. 1) for each attention event in state space Ω ,  1,,0 ≤≤ CBA ppp .  2) for the whole attention eventΩ ,  1)( =ΩP .  3) for mutually exclusive attention event,  , ,, ,( ) ii A B Ci A B CP i p=== ∑∪ .  Let the initial condition,   [ ] [ ]0,1,1)0(),0(),0( =CBA ppp ,   And then, Laplace transform is employed for solving probability shifting model.    3. Factor Analysis  More than 60 percent of all aircraft accidents are concerned to pilot error. Identifying when equipment and procedures do not fully support the operational needs of pilots is critical to reducing error and improving flight safety [21]. Therefore, to develop new flight deck technologies that have traditionally followed a design process more focused on component functionality and technical performance than pilot usage and operability is  especially important. For a number of tasks, the pilots’ visual attention is divided between tracking one or more vehicles and scanning the environment for information. Serious piloting errors and accidents are rare events and the low-probability of occurrence makes the study of pilot error difficult to investigation.  For contributing to counter these errors and accidents, we simplify these troubles as follows. Pilots’ visual attention is conveyed by state vector such as visual tracking, visual interference and  visual dormancy.  In this paper we present a method for making short-term predictions about the visual attention shifting. The motivation to develope a method for anticipating where to look is rooted in the need for visual attention in virtual pilots. Next, the pilots’ visual attention data are used to illustrate the performance of the probability  shifting model. There are so many factors affect the pilot in the aircraft flying, but the experience and sex play an important role. For investigating the effect of driving experience and sexual distinction on pilots’ visual attention, the volunteer is split up into two groups for this experiment. Firstly, the experiment is launched between male pilots and female pilots. And then, similar experiment is launched between experienced pilots and inexperienced pilots.   4. Analyze to Driving Experience  For the purposes of researching the effect of driving experience on pilots’ visual attention, the probability of visual tracking, visual interference and visual dormancy for different experienced and inexperienced pilots are calculated.   4.1. Experiment 1: Experienced Pilots  Fig. 1 illustrates the probability of state vector including visual tracking, visual interference and visual dormancy for experienced pilots. We observe that the probability of visual tracking is far larger Sensors & Transducers, Vol. 158, Issue 11, November 2013, pp. 357-362  360than the sum of visual interference and  visual dormancy. For experienced pilots, the probability of visual tracking (black) 95.0>Ap  in 50 minutes. The probability of visual interference (red) Bp  and visual dormancy (blue) Cp  are very little by comparison.   0 50 100 150 200 250 30000.10.20.30.40.50.60.70.80.91  Fig. 1. The probability of visual tracking (black), visual interference (red) and visual dormancy (blue)  for experienced pilots.   4.2. Experiment 2: Inexperienced Pilots  Fig. 2 illustrates the probability of visual tracking, visual interference and visual dormancy for inexperienced pilots. The cited experimental curves analogously with the probability of  experienced pilots.   0 50 100 150 200 250 30000.10.20.30.40.50.60.70.80.91  Fig. 2. The probability of visual tracking (black), visual interference (red) and visual dormancy (blue) for inexperienced pilots.  Despite similar signs of the probability for visual tracking, visual interference and visual dormancy between inexperienced pilots and experienced pilots when 50<t  minutes, the probability of visual interference for inexperienced pilots is greater than it for experienced pilots when 150>t  minutes. This shows that the capacity of resisting disturbance is becoming stronger and stronger with experience.   5. Analyze to Sexual Distinction  For the purposes of researching the effect of sexual distinction on pilots’ visual attention, the probability of visual tracking, visual interference and visual dormancy for different sex pilots are calculated.   5.1. Experiment 3: Male Pilots  The probability of state vector including visual tracking, visual interference and visual dormancy for male pilots is shown in Fig. 3. That the probability of visual tracking is far larger than the sum of visual interference and visual dormancy is in keeping with the real conditions. For male pilots, the probability of visual tracking (black) 95.0>Ap  in 60 minutes, on the contrary, the probability of visual interference (red) Bp  and visual dormancy (blue) Cp  are less than 02.0 .   0 50 100 150 200 250 30000.10.20.30.40.50.60.70.80.91  Fig. 3. The probability of visual tracking (black), visual interference (red) and visual dormancy (blue) for male pilots.   5.2. Experiment 4: Female Pilots  For researching the visual attention of female pilots’ visual attention, the probability of visual tracking, visual interference and visual dormancy for female pilots are displayed in Fig. 4. Apparently, the Sensors & Transducers, Vol. 158, Issue 11, November 2013, pp. 357-362  361probability of visual tracking (black) 95.0>Ap about 50 minutes, which is less than the probability of male pilots.   0 50 100 150 200 250 30000.10.20.30.40.50.60.70.80.91  Fig.  4. The probability of visual tracking (black), visual interference (red) and visual dormancy (blue)  for female pilots.   By comparing Fig. 3 and Fig. 4, we can find that the probability of visual tracking, visual interference and visual dormancy of female pilots are similar to male pilots. The forecast of the probability shifting model is coincident with the fact. The results of this section suggest the use of the probability shifting model as a powerful technique for measuring the movement of pilots’ visual attention.   4. Conclusions  The probability shifting model is applied to the pilots’ visual attention data in order to investigate the movement of visual attention. It is found that shifting model can provide very useful information about possible movement of pilots’ visual attention. Moreover, we can find that the forecast of the model agree with the actual facts by investigating the effect of driving experience and sexual distinction on pilots’ visual attention. It might be seen that the method presented in the paper is of practicality and reliability. In conclusion, we have reported here on exploratory results that demonstrate effective and potential usefulness of probability-based techniques.   Acknowledgements  The financial support from the funds of the Fundamental Research Funds for the Central Universities under Grant No. ZXH 2012C002, the National Natural Science Foundation of China under Grant No. U1233201 are gratefully acknowledged.  References  [1]. M. Yang, Y. Wu, S. Lao, Intelligent collaborative tracking by mining auxiliary objects, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA, 17-22 June 2006, pp. 697–704. [2]. H. Liu, L. Zhang, Z. Yu, H. Zha, Y. Shi, Collaborative mean shift tracking based on multi-cue integration and auxiliary objects, in Proceedings of the International Conference on Image Processing, San Antonio, USA, 16-19 September 2007,  pp. 217–220. [3]. B. J. Scholl, Objects and attention: the state of the art, Cognition, Vol. 80, Issue 1-2, 2001, pp. 1–46. [4]. A. M. Treisman, G. Gelade, A feature-integration theory of attention, Cognitive Psychology, Vol. 12, Issue 1, 1980, pp. 97–136. [5]. J. M. Wolfe, Visual search in continuous naturalistic stimuli, Vision Research, Vol. 34, Issue 9, 1994, pp. 1187–1195. [6]. L. Itti, E. Niebur, C. Koch, A model of saliency-based fast visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, Issue 11, 1998, pp. 1254–1259. [7]. H. Liu, Y. Shi, Robust visual tracking based on selective attention shift, in Proceedings of the IEEE International Symposium on Intelligent Control, Saint Petersburg, Russia, 2009, pp. 1176–1179.  [8]. J. Reynolds, L. Chelazzi, R. Desimone, Competitive mechanisms subserve attention macaque areas V2 and V4, Journal of Neuroscience, Vol. 19, Issue 5, 1999, pp. 1736–1753. [9]. J. Reynolds, D. Heeger, The normalization model of attention, Neuron, Vol. 61, Issue 2, 2009,  pp. 168–185. [10]. J. Lee, J. H. Maunsell, A normalization model of attentional modulation of single unit responses, PLoS One, Vol. 4, Issue 1, 2008, e4651. [11]. S. Shioiri, K. Yamamoto, Y. Kageyama, H. Yaguchi, Smooth shifts of visual attention, Vision Research, Vol. 42, 2002, pp. 2811–2816. [12]. J. F. Mitchell, K. A. Sundberg, J. H. Reynolds, Differential attention-dependent response modulation across cell classes in macaque visual area V4, Neuron, Vol. 55, 2007, pp. 131–141. [13]. M. Raffi, S. Squatrito, M. G. Maioli, Gaze and smooth pursuit signals interact in parietal area 7 m of the behaving monkey, Experimental Brain Research, Vol. 182, 2007, pp. 35–46. [14]. G. M. Ghose, J. H. Maunsell, Spatial summation can explain the attentional modulation of neuronal responses to multiple stimuli in area V4, Journal of Neuroscience, Vol. 28, 2008, pp. 5115–5126. [15]. P. S. Khayat, R. Niebergall, J. C. Martinez-Trujillo, Attention differentially modulates similar neuronal responses evoked by varying contrast and direction stimuli in area MT, Journal of Neuroscience, Vol. 30, Issue 6, 2010, pp. 2188–2197. [16]. G. J. Zelinsky, D. L. Sheinberg. Eye movements during parallel/serial visual search, Journal of Experimental Psychology: Human Perception and Performance, Vol. 23, Issue 1, 1997,  pp. 244-262. [17]. G. J. Zelinsky, R. P. N. Rao, M. M. Hayhoe, D. H. Ballard, Movements reveal the spatio-temporal dynamics of visual search, Psychological Science, Vol. 8, Issue 6, 1996, pp. 448-453. Sensors & Transducers, Vol. 158, Issue 11, November 2013, pp. 357-362  362[18]. S. L. Yeh, I. P. Chen, K. K. De Valois, R. L. De Valois, Figural aftereffects and spatial attention, Journal of Experimental Psychology: Human Perception and Performance, Vol. 22, Issue 2, 1996, pp. 446-460. [19]. B. J. Avolio, K. G. Kroeck, P. E. Panek, Individual differences in information processing ability as a predictor of motor vehicle accident, Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 27, Issue 5, 1985, pp. 577–587. [20]. N. E. Laurie, S. Zhang, R. Mundoli, S. A. Duffy, An evaluation of alternative do not enter signs: failures of attention, Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 7, Issue 3, 2004, pp. 151–166. [21]. D. C. Foyle, B. L. Hooey, M. D. Byrne, K. M. Corker, S. Deutsch, C. Lebiere, K. Leiden, C. Wickens, Human performance models of pilot behavior, in Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, Orlando, USA, 26-30 September 2005, pp. 1109-1113.   ___________________  2013 Copyright ©, International Frequency Sensor Association (IFSA). All rights reserved. (http://www.sensorsportal.com) 

The Movement Model of Pilots' Visual Attention

http://www.sensorsportal.com/HTML/DIGEST/november_2013/PDF_vol_158/P_1511.pdf

The Movement Model of Pilots' Visual Attention

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals