An Analysis of Concentration Region on Powerpoint Slides using Eye Tracking

Powerpoint slides have become one of the essential teaching  tools in academic  for both  offline and  online modes. It may  play  a  useful  role to facilitate  discussion  and  information exchange.  However,  in  our  teaching  experience,  we find  many students utilizing Powerpoint slides beyond their traditional functions.  Many  students   fully  rely on  the  slides  as  the  main learning   materials   and,  in  some  cases,  substituting  textbooks. This  study  intends  to  understand how  students   interact   with the   learning   materials   presented  on  Powerpoint   slides.  The interaction is measured using  an  eye tracker device called  the Eye  Tribe  Tracker. Thirty  sophomore  and  junior  students  are asked  to participate. They are  instructed to learn  a topic in the subject  of Introduction to Algorithm  and  Programming, a basic course  in the  computer science field. During  the  process,  their fixation points  are  monitored  and  are  related to the contents  on the slides. The results are rather surprising. Many students  read the  slides  in  unexpected  manners that  may  compromise   their understanding and  may lead to inaccurate interpretations

Relating Eye-Tracking Measures With Changes In Knowledge on Search Tasks

We conducted an eye-tracking study where 30 participants performed searches on the web. We measured their topical knowledge before and after each task. Their eye-fixations were labelled as "reading" or "scanning". The series of reading fixations in a line, called "reading-sequences" were characterized by their length in pixels, fixation duration, and the number of fixations making up the sequence. We hypothesize that differences in knowledge-change of participants are reflected in their eye-tracking measures related to reading. Our results show that the participants with higher change in knowledge differ significantly in terms of their total reading-sequence-length, reading-sequence-duration, and number of reading fixations, when compared to participants with lower knowledge-change.Comment: ACM Symposium on Eye Tracking Research and Applications (ETRA), June 14-17, 2018, Warsaw, Polan

An Evaluation of Mouse and Keyboard Interaction Indicators towards Non-intrusive and Low Cost Affective Modeling in an Educational Context

AbstractIn this paper we propose a series of indicators, which derive from user's interactions with mouse and keyboard. The goal is to evaluate their use in identifying affective states and behavior changes in an e-learning platform by means of non-intrusive and low cost methods. The approach we have followed study user's interactions regardless of the task being performed and its presentation, aiming at finding a solution applicable in any domain. In particular, mouse movements and clicks, as well as keystrokes were recorded during a math problem solving activity where users involved in the experiment had not only to score their degree of valence (i.e., pleasure versus displeasure) and arousal (i.e., high activation versus low activation) of their affective states after each problem by using the Self-Assessment-Manikin scale, but also type a description of their own feelings. By using that affective labeling, we evaluated the information provided by these different indicators processed from the original user's interactions logs. In total, we computed 42 keyboard indicators and 96 mouse indicators

Analyse visuelle et cérébrale de l’état cognitif d’un apprenant

Un état cognitif peut se définir comme étant l’ensemble des processus cognitifs inférieurs (par exemple : perception et attention) et supérieurs (par exemple : prise de décision et raisonnement), nécessitant de la part de l’être humain toutes ses capacités mentales en vue d’utiliser des connaissances existantes pour résoudre un problème donné ou bien d’établir de nouvelles connaissances. Dans ce contexte, une attention particulière est portée par les environnements d’apprentissage informatisés sur le suivi et l’analyse des réactions émotionnelles de l’apprenant lors de l’activité d’apprentissage. En effet, les émotions conditionnent l’état mental de l’apprenant qui a un impact direct sur ses capacités cognitives tel que le raisonnement, la prise de décision, la mémorisation, etc. Dans ce contexte, l’objectif est d’améliorer les capacités cognitives de l’apprenant en identifiant et corrigeant les états mentaux défavorables à l’apprentissage en vue d’optimiser les performances des apprenants. Dans cette thèse, nous visons en particulier à examiner le raisonnement en tant que processus cognitif complexe de haut niveau. Notre objectif est double : en premier lieu, nous cherchons à évaluer le processus de raisonnement des étudiants novices en médecine à travers leur comportement visuel et en deuxième lieu, nous cherchons à analyser leur état mental quand ils raisonnent afin de détecter des indicateurs visuels et cérébraux permettant d’améliorer l’expérience d’apprentissage. Plus précisément, notre premier objectif a été d’utiliser les mouvements des yeux de l’apprenant pour évaluer son processus de raisonnement lors d’interactions avec des jeux sérieux éducatifs. Pour ce faire, nous avons analysé deux types de mesures oculaires à savoir : des mesures statiques et des mesures dynamiques. Dans un premier temps, nous avons étudié la possibilité d’identifier automatiquement deux classes d’apprenants à partir des différentes mesures statiques, à travers l’entrainement d’algorithmes d’apprentissage machine. Ensuite, en utilisant les mesures dynamiques avec un algorithme d’alignement de séquences issu de la bio-informatique, nous avons évalué la séquence logique visuelle suivie par l’apprenant en cours de raisonnement pour vérifier s’il est en train de suivre le bon processus de raisonnement ou non. Notre deuxième objectif a été de suivre l’évolution de l’état mental d’engagement d’un apprenant à partir de son activité cérébrale et aussi d’évaluer la relation entre l’engagement et les performances d’apprentissage. Pour cela, une étude a été réalisée où nous avons analysé la distribution de l’indice d’engagement de l’apprenant à travers tout d’abord les différentes phases de résolution du problème donné et deuxièmement, à travers les différentes régions qui composent l’interface de l’environnement. L’activité cérébrale de chaque participant a été mesurée tout au long de l’interaction avec l’environnement. Ensuite, à partir des signaux obtenus, un indice d’engagement a été calculé en se basant sur les trois bandes de fréquences α, β et θ. Enfin, notre troisième objectif a été de proposer une approche multimodale à base de deux senseurs physiologiques pour permettre une analyse conjointe du comportement visuel et cérébral de l’apprenant. Nous avons à cette fin enregistré les mouvements des yeux et l’activité cérébrale de l’apprenant afin d’évaluer son processus de raisonnement durant la résolution de différents exercices cognitifs. Plus précisément, nous visons à déterminer quels sont les indicateurs clés de performances à travers un raisonnement clinique en vue de les utiliser pour améliorer en particulier, les capacités cognitives des apprenants novices et en général, l’expérience d’apprentissage.A cognitive state can be defined as a set of inferior (e.g. perception and attention) and superior (e.g. perception and attention) cognitive processes, requiring the human being to have all of his mental abilities in an effort to use existing knowledge to solve a given problem or to establish new knowledge. In this context, a particular attention is paid by computer-based learning environments to monitor and assess learner’s emotional reactions during a learning activity. In fact, emotions govern the learner’s mental state that has in turn a direct impact on his cognitive abilities such as reasoning, decision-making, memory, etc. In this context, the objective is to improve the cognitive abilities of the learner by identifying and redressing the mental states that are unfavorable to learning in order to optimize the learners’ performances. In this thesis, we aim in particular to examine the reasoning as a high-level cognitive process. Our goal is two-fold: first, we seek to evaluate the reasoning process of novice medical students through their visual behavior and second, we seek to analyze learners’ mental states when reasoning to detect visual and cerebral indicators that can improve learning outcomes. More specifically, our first objective was to use the learner’s eye movements to assess his reasoning process while interacting with educational serious games. For this purpose, we have analyzed two types of ocular metrics namely, static metrics and dynamic metrics. First of all, we have studied the feasibility of using static metrics to automatically identify two groups of learners through the training of machine learning algorithms. Then, we have assessed the logical visual sequence followed by the learner when reasoning using dynamic metrics and a sequence alignment method from bio-informatics to see if he/she performed the correct reasoning process or not. Our second objective was to analyze the evolution of the learner’s engagement mental state from his brain activity and to assess the relationship between engagement and learning performance. An experimental study was conducted where we analyzed the distribution of the learner engagement index through first, the different phases of the problem-solving task and second, through the different regions of the environment interface. The cerebral activity of each participant was recorded during the whole game interaction. Then, from the obtained signals, an engagement index was computed based on the three frequency bands α, β et θ. Finally, our third objective was to propose a multimodal approach based on two physiological sensors to provide a joint analysis of the learner’s visual and cerebral behaviors. To this end, we recorded eye movements and brain activity of the learner to assess his reasoning process during the resolution of different cognitive tasks. More precisely, we aimed to identify key indicators of reasoning performance in order to use them to improve the cognitive abilities of novice learners in particular, and the learning experience in general

Eye Tracking: A Promising Means of Tracing, Explaining, and Preventing the Effects of Display Clutter in Real Time.

Display clutter is a widely-acknowledged but ill-defined problem that affects operators in complex, data-rich domains, such as medicine and aviation. Largely regarded a function of data density and display organization, clutter has been shown to degrade performance on a range of tasks, most notably visual search and noticing. Clutter effects may be exacerbated by stress, a major performance-shaping factor in the above domains. The goal of this dissertation was to develop an eye tracking-based approach for tracing and preventing the effects of clutter and stress on attention allocation and information acquisition. The research involved three stages: 1) identify the most diagnostic eye tracking metrics for capturing and explaining the effects of clutter and stress on performance, 2) determine which eye tracking metrics can detect the effects of clutter early on, in real time, and form the basis for models of clutter effects, and 3) evaluate the effectiveness of real-time display adjustments for preventing performance decrements. This research was carried out in several contexts, including emergency department (ED) electronic medical records (EMRs). First, three experiments were conducted in different application domains, including the ED, to establish the relationship between clutter, stress, attention, and performance during visual search and noticing tasks. Clutter resulted in performance decrements on both tasks. The underlying changes in attention allocation were captured by several eye tracking metrics, some of which were able to differentiate between the effects of data density and organization. A fourth experiment calculated the most promising eye tracking metrics in real time and used them as input to logistic regression models of response time. Long response time due to poor organization could be modeled most accurately. Finally, a fifth experiment presented ED physicians with real-time adaptations (highlighting and shortcut panel) to their EMR while they reviewed patient records to perform diagnoses. Both adjustments led to better performance and were viewed favorably by physicians. Overall, this research adds to the knowledge base on clutter and visual attention, supports the further development of eye tracking as a basis for real-time processing, and contributes to improved safety in various domains by supporting timely and accurate information acquisition.PhDIndustrial and Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/113627/1/nadmarie_1.pd

Eye Tracking to Support eLearning

Online eLearning environments to support student learning are of growing importance. Students are increasingly turning to online resources for education; sometimes in place of face-to-face tuition. Online eLearning extends teaching and learning from the classroom to a wider audience with different needs, backgrounds, and motivations. The one-size-fits-all approach predominately used is not effective for catering to the needs of all students. An area of the increasing diversity is the linguistic background of readers. More students are reading in their non-native language. It has previously been established that first English language (L1) students read differently to second English language (L2) students. One way of analysing this difference is by tracking the eyes of readers, which is an effective way of investigating the reading process. In this thesis we investigate the question of whether eye tracking can be used to make learning via reading more effective in eLearning environments. This question is approached from two directions; first by investigating how eye tracking can be used to adapt to individual student’s understanding and perceptions of text. The second approach is analysing a cohort’s reading behaviour to provide information to the author of the text and any related comprehension questions regarding their suitability and difficulty. To investigate these questions, two user studies were carried out to collect eye gaze data from both L1 and L2 readers. The first user study focussed on how different presentation methods of text and related questions affected not only comprehension performance but also reading behaviour and student perceptions of performance. The data from this study was used to make predictions of reading comprehension that can be used to make eLearning environments adaptive, in addition to providing implicit feedback about the difficulty of text and questions. In the second study we investigate the effects of text readability and conceptual difficulty on eye gaze, prediction of reading comprehension, and perceptions. This study showed that readability affected the eye gaze of L1 readers and conceptual difficulty affected the eye gaze of L2 readers. The prediction accuracy of comprehension was consequently increased for the L1 group by increased difficulty in readability, whereas increased difficulty in conceptual level corresponded to increased accuracy for the L2 group. Analysis of participants’ perceptions of complexity revealed that readability and conceptual difficulty interact making the two variables hard for the reader to disentangle. Further analysis of participants’ eye gaze revealed that both the predefined and perceived text complexity affected eye gaze. We therefore propose using eye gaze measures to provide feedback about the implicit reading difficulty of texts read. The results from both studies indicate that there is enormous potential in using eye tracking to make learning via reading more effective in eLearning environments. We conclude with a discussion of how these findings can be applied to improve reading within eLearning environments. We propose an adaptive eLearning architecture that dynamically presents text to students and provides information to authors to improve the quality of texts and questions