Attention is strongly connected with learning and when it comes to
acquiring new knowledge, attention is one the most important mechanisms. The
learner’s attention affects learning results and can define the success or failure
of a student. The negative effects are especially significant when carrying out
long or demanding tasks, as often happens in an assessment. This paper presents
a monitoring system using computer peripheral devices. Two classes were
monitored, a regular one and an assessment one. Results show that it is possible
to measure attentiveness in a non-intrusive way.This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and
FCT – Fundação para a Ciência e Tecnologia within the Project Scope:
UID/CEC/00319/2013.info:eu-repo/semantics/publishedVersio

Bajo, Javier

Carneiro, Davide Rua

Durães, Dalila

Novais, Paulo

Universidade do Minho: RepositoriUM

adfa, p. 1, 2011. © Springer-Verlag Berlin Heidelberg 2011 Using Computer Peripheral Devices to Measure Attentiveness Dalila Durães1, Davide Carneiro2,3, Javier Bajo1, Paulo Novais2 1Department of Artificial Intelligence, Technical University of Madrid, Madrid, Spain d.alves@alumnos.upm.es, jbajo@fi.upm.es 2Algoritmi Center, Minho University, Braga, Portugal dcarneiro@di.uminho.pt, pjon@di.uminho.pt 3CIICESI, ESTGF, Polytechnic Institute of Porto, Portugal Abstract. Attention is strongly connected with learning and when it comes to acquiring new knowledge, attention is one the most important mechanisms. The learner’s attention affects learning results and can define the success or failure of a student. The negative effects are especially significant when carrying out long or demanding tasks, as often happens in an assessment. This paper pre-sents a monitoring system using computer peripheral devices. Two classes were monitored, a regular one and an assessment one. Results show that it is possible to measure attentiveness in a non-intrusive way. Keywords: Attentiveness, Learning Activities, Mental Fatigue, Stress. 1 Introduction Our society, in permanent change, requires a continuous adaptation of the human being to the surrounding environment. We live in a global, multicultural and hyper-connected world where technology is present in all spheres of life and is the backbone for the transformation of society. In the education field, ICT should be considered as an innovation, since it involves a personal and collective work of reflection, realization and change. So ICT has ex-panded the range of possibilities of teaching and implementing innovative methodol-ogies.  Scientific studies have shown the influence of states attention on student learning [1-3]. These indicators are useful to predict the behavior of a student and identify potential problems in the course of their learning. By using behavioral biometrics, especially Keystroke and Mouse Dynamics, it’s possible analyze the type of the task performed by each user, the time spent performing it, as well as the mental workload of the task. With this information it’s also possible for classifiers to distinguish situa-tions in which each user shows signs of attentiveness and where higher or lower men-tal workload is measured [1]. This technique is based only on the observation of the use of the mouse and the keyboard, which allows an assessment of the user’s performance. Consequently, we considered both non-invasive and non-intrusive approaches. In this way we can use this technique to develop the attention level and management initiatives in the context of learning activities, allowing teachers to perceive the student’s state of mind and adjust the teaching process to the student’s needs and behavior. Teaching should be solidly grounded to the absolute understanding of how the pro-cess of learning occurs so that instructional strategies could be efficient and lead to persistent knowledge. This is especially true when learning activities involve technol-ogies. In such cases, some of the previously mentioned issues can negatively affect the acquisition of knowledge and the persistence of that same knowledge since stu-dents have other technologies and applications that they can use and this can distract them. In this article we focus on a new field of application of ICT techniques and tech-nologies in learning activities.  The goal is to determine the level of attentiveness in normal and assessment lessons and compare it with results achieved by students. This paper is organized as follows. In the next Section theoretical foundations of attention where scientific literature is reviewed, Section 3 analyzes keyboard and mouse for attentiveness detection. Finally, in Section 4 and 5 some initial results and conclusions of this work are presented.  2 Theoretical Foundations The concept of attention has had different definitions since the nineteenth century. Initially, it was only a study field of psychology. However, in recent years it was ob-ject of study in different areas including biology, education science, psychiatry, and computer science. Being a cognitive process, attention is strongly connected with learning [2]. When it comes to acquiring new knowledge, attention can be considered one of the most important mechanisms [1]. The level of the learner’s attention affects learning results. The lack of attention can define the success of a student. In learning activities, atten-tion is also very important to perform these tasks in an efficient and adequate way.  When students are using technology, and especially when connected to the Inter-net, distractions can occur. This happens because they have access to messages from chats, social networks and emails; and other applications like music applications and news sites, which can be more attractive to students. Moreover, these applications can constantly run in the background. When these activities are prolonged for a long peri-od they may have decreased the level of attention [4]. 2.1 Features that influence attention Generally, there are some factors that influence attention level: stress, mental fatigue, anxiety, emotions, different environment and human health [16].  Stress may have a positive or negative influence. On the one hand it is generally accepted that stressful events increase the level of attention [5 - 9]. On the other hand, there are cases in which stressful events cause depression or aging [10, 11]. When some activities are prolonged for a long period of time, our brain may feel overheaded with such amount of information, and this leads to a potential emergence of mental fatigue, which decreases the level of students’ attention. A substantial literature shows that anxiety affects perceptual and related processes of attention [12]. Anxiety has an impact on cognition and attention because it is often associated with adverse effects on attention of cognitive tasks [13].  Finally, health problems, mood, and the surrounding environment can also influ-ence the level of attentiveness. Figure 1 presents a design of factors that influence attention.   Fig. 1. Conceptualization of the set of factors that influence attention. 3 Study Outline The purpose of this work is to compare a normal and an assessment class at the Sec-ondary School of Caldas das Taipas, Guimarães, Portugal. We want to determine if classes with different goals have a significant effect on mouse and keyboard dynamics and how can we estimate attention level. 3.1 Methodology For this purpose, a group of 13 (10 girls and 3 boys) art students were selected to participate, whose average age is seventeen years old. In different weeks, they have a normal and an assessment lesson, where they have access to an individual computer and three hours to complete a task. The lesson started at 8:30 and finished at 11:00 a.m. Students received, at the beginning of the lesson, a document with the goals of the task. The normal and the assessment lessons contained tasks to be completed us-ing Photoshop. Data collection was carried out using a logger application developed in previous work [4]. The data collected by the logger application, characterizing the students’ interaction patterns, is aggregated in a server to which the logger application connects after the student logs in. This application runs in the background, which makes the data acquisition process, a completely transparent one, from the point of view of the student. 3.2 Data Analysis In this section we show the existence of different behaviors in the two different les-sons. Data was analyzed in two different ways. First was carried out a general analy-sis in which statically methods are used to obtain preliminary conclusions. Second, an individual analysis was made in order to compare the different moments.  Although the collected data describes the interaction with both the mouse and the keyboard [14], only data from the mouse was considered in this analysis. This is due to the characteristics of the task, which was based on Photoshop that requires mostly the interaction with the mouse The amount of data collected from the keyboard was too small to allow sound analyses. Another important aspect worth mentioning is that Photoshop requires a precise use of the mouse, which makes it a suitable application to the current study. In a preliminary analysis of the data, we concluded that there are indeed different interaction patterns depending on the type of lesson analyzed indeed. To conclude this, we looked at the distributions of the data collected and analyzed the statistical significance of their differences. To this end, we used the Kruskal-Wallis test. Table 1 details the mean value of each feature in each class (evaluation and normal). It also details the p-value of the Kruskal-Wallis test.  Feature Symbol Significance Mean Evaluation Mean NormalMouse Velocity mv 0,0011 0,49 0,53Mouse Aceleration ma 0,0010 0,54 0,57Click Duration cd 0,0035 245,38 159,83Time Between Clicks tbc 0,0966 1964,19 3063,11Distance Between Click dbc 0,0001 150,83 206,12Duration Distance Clicks ddc 0,0545 143,04 143,97Excess Distance Between Clicks edbc 0,0000 154,00 309,48Absolute Excess Distance Between Click aedbc 0,0000 1,54 2,05Absolute Sum Distance Between Clicks asdbc 0,0094 4006,31 5038,24Distance Point to Line Between Clicks dplbc 0,0169 21611000,00 37026200,00Absolute Distance Point Between Clicks adpbc 0,1361 157647,00 208223,00Table 1. Results of the Kruskall-Wallis test and mean values for ech class and each feature. When data from the two classes is compared, the first conclusion is that the differ-ences observed are statistically significant in nearly all features, with the exception of Time Between Clicks, Distance During Clicks, and Average Distance Point Between Clicks.  Moreover, mean values of the features are consistently lower in the evaluation class. In most of the features, this indicates an increased performance (e.g. a smaller average distance between clicks means that the student moved the mouse in a more efficient manner). However, in the case of mouse velocity, for example, a smaller velocity could point out a slower, and thus less efficient, movement. In past work we concluded that a slower mouse velocity is indeed necessary for the student to achieve increased accuracy in mouse movement: moving the mouse too fast would make pre-cise movements more difficult to carry out. This is especially true in tasks such as those of this study. A similar trend happens with mouse acceleration and click dura-tion. The remaining features consistently show increased performance in the evalua-tion class. The Figure 2 shows the distribution of the values for all the users and for each fea-ture. We can see that the results of each feature are very different in the two lessons. The students react in different ways in the assessment and normal lesson. We can also consider that, in general, they are more focused on the assessment lesson because they have a mouse velocity, mouse acceleration, and distance point to line between clicks slower and the click duration was higher.   Fig. 2. Distributions of the data collected in the two different classes (Evaluation vs. Normal), in four different features: Mouse Velocity, Mouse Acceleration, Click Duration and Distance of the Pointer do the Line Between Clicks. The selected of features characterize several aspects of interaction on normal and assessment lesson. However, this doesn’t mean that they are all affected equally or that they are all affected at all when they have an assessment. A curious property of these features comparing the normal and the assessment lesson was that the mouse velocity and the mouse acceleration are lower and the click duration is higher in the assessment lesson. Figure 3 compares the histograms of mv, ma, and cd and evidenc-es the differences between the two classes.  Fig. 3. Comparing mv, ma, and cd during the lessons. From these studies we conclude that the most significantly affected features are mv, ma, cd, tbc, dbc, ddc, edbc, ssdbc, and dplbc. In all these features almost all students show statistically significant differences when comparing the two lessons.  4 Preliminary Results  During the two lessons, the monitoring system was used to assess the interaction of the students with the computer and to quantify their level of attentiveness, as well. To quantify attentiveness the following methodology was followed. Asides from captur-ing the interaction of the students with the computer, the monitoring system also reg-isters the applications with which students are interacting. We analyzed all the appli-cations used by all students and labelled each one of them as belonging to the task or not. We then quantify the amount of time that each student spends interacting with applications related to the task versus other applications. Table 2 details the results of this quantification. Students have a clearly different attitude in the two lessons: when being evaluated, they spend more time interacting with task-related applications, a sign that they are more focused on the task. The re-sults thus point out, not only, that it is possible to quantify student attentiveness in a non-intrusive way, but also that attention is higher when under evaluation. This was an expected conclusion but it nonetheless validates the proposed approach. We also analyzed the correlation between the level of attentiveness and the score of the students in the task. There is a weak positive correlation (0.41) between the two variables. We believe that this value is not higher for two reasons: (1) this class is mostly composed by “excellent” students and (2) the task was of average difficulty. Thus, there were not many differences in the scores. In future work we will study this relationship in more detail, namely in different classes and with tasks of different levels of difficulty. Total Time (s) % Time Total Time (s) % TimeT7110001 3156,22 67% 690,20 31%T7110003 2416,01 55% 1253,07 32%T7110005 2475,14 63% 1807,26 40%T7110006 1571,11 36% 494,07 19%T7110007 3177,28 53% 701,26 30%T7110008 2492,51 58% 783,33 26%T7110009 4264,73 72% 2130,49 53%T7110010 3239,52 74% 1451,13 40%T7110011 3845,71 71% 737,86 20%T7110012 3581,57 58% 157,73 30%T7110013 835,24 48% 1588,86 40%StudentAssessment NormalTable 2. Total time (seconds) devoted to the task and percentage of total time devoted to the task, while being assessed and while in a normal class, for each student. 5 Discussion and Future Work It is imperative to mention that these features aim to quantify the student’s work and estimate the level of attentiveness. These experiments allow drawing some interesting conclusions about students and their behavior during the assessment. However, this test was implemented in a good class with higher scores in all subjects. Yet, they react in different ways during normal or assessment lessons. We can see that they are more focused on the assessment lesson so they have a higher attention level in these situa-tions. We also concluded that there are also significant differences in the interaction patterns when comparing the two classes.  In future work we will determine if there is a relationship between these two phe-nomena or if they are both dependent on other factors, such as the level of stress. We will also implement similar studies in other classes with overall worse scores and using tasks of different complexities. With this we will train classifiers that can esti-mate the level of attention of students, in real-time, to provide an important source of information for teachers to act accordingly and in a timely manner [15].  Acknowledgements This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT – Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2013. References 1. Pimenta, A., Gonçalves, S., Carneiro, D., Riverola, F., Novais, P. (2014). Mental Work-load Management as a Tool in e-Learning Scenarios. 2. Hwang, K., Yang, C. (2009). Automated Inattention and Fatigue Detection System in Dis-tance Education for Elementary School Students. Journal of Educational Technology 12 pp. 22-35. 3. Esysench, M.W. (1992). Anxiety: The cognitive Perspective. Hove, England: Erlbaum. 4. Pimenta, A., Carneiro D., Neves, J., Novais P. (2015). A Neural Network to Classify Fa-tigue from Human-Computer Interaction. Neurocomputing. vol.172, pp. 413-426. 5. Olff, M. et al. (2005). The Psychobiology of PTSD: coping with trauma. Psichoneuroen-docrinology 30, 974-982. 6. Oitzl, M.S. and Kloet, E.R. (1992). Selective corticosteroid antagonists  modulate specific aspects of spatial orientation learning. Behav. Neurosci. 106, 62-71. 7. Sandi, C. and Rose, S.P. (1994). Corticosterone enhances log-term retention in one-day-old chicks trained in a weak passive avoidance learning paradigm. Brain. Res. 647, 106-112. 8. Sandi, C. (1997). Experience-dependence facilitating effect of corticosterone on spatial memory formation in the water maze. Eur. J. Neurosci. 9, 637-642. 9. Roosendaal, B. and MAcGaugh, J.L.(1996). Amygdaloid nuclei lesions differentially af-fect glucorticoid-induced memory enhancement in a inhibitory avoidance. Task. Neurobi-ol. Learn. Mem. 65, 1-8. 10. Shors, T.J. Stressfull experience and Learning across Lifespan. Annu. Rev. Psychol. (in press). 11. McGaugh, J.L. (2004). The Amygdala Modulates the Consolidation of Memories of Emo-tionally arousing Experiences. Annu. Rev. Neurosci. 27, 1-28. 12. Bishop, S. (2008). Trait Anxiety and Impoverished Prefrontal Control of Attention. Nature Neuroscience 12, pp. 92-98. 13. van der Linden, D., Frese, M., Meijman, T.F. (2003). Mental Fatigue and the Control of Cognitive Processes: Effects on Perseveration and Planning. Acta Psychologica 113(1) pp. 45-65. 14. Carneiro, D., Novais, P., Pêgo, J.M., Sousa, N., Neves, J. (2015). Using Mouse Dynamics to Assess During Online Exams. Hybrid Artificial Intelligent Systems, vol. 9121, pp. 345-356. 15. Lima, L., Novais, P., Costa, R., Cruz, J. B., & Neves, J. (2010). Group decision making and Quality-of-Information in e-Health systems. Logic Journal of IGPL, jzq029. 16. Pimenta, A., Carneiro, D., Novais, P., & Neves, J. (2013). Monitoring mental fatigue through the analysis of keyboard and mouse interaction patterns. In Hybrid Artificial Intel-ligent Systems (pp. 222-231). Springer Berlin Heidelberg.    

Using computer peripheral devices to measure attentiveness

Carneiro, Davide

Repositório Científico do Instituto Politécnico do Porto

Using Computer Peripheral Devices to Measure Attentiveness

http://repositorium.sdum.uminho.pt/bitstream/1822/50563/1/2016_Using%20Computer%20Peripheral%20Devices%20to%20Measure%20Attentiveness%20-%20final%20version.pdf

Using computer peripheral devices to measure attentiveness

Abstract

Similar works

Full text

Available Versions

Universidade do Minho: RepositoriUM

Repositório Científico do Instituto Politécnico do Porto