Accuracy and precision of fixation locations recorded with the low-cost Eye Tribe tracker in different experimental set-ups

This article compares the accuracy and precision of the low-cost Eye Tribe tracker and a well-established comparable eye tracker: SMI RED 250. Participants were instructed to fixate on predefined point locations on a screen. The accuracy is measured by the distance between the recorded fixation locations and the actual location. Precision is represented by the standard deviation of these measurements. Furthermore, the temporal precision of both eye tracking devices (sampling rate) is evaluated as well. The obtained results illustrate that a correct set-up and selection of software to record and process the data are of utmost importance to obtain acceptable results with the low-cost device. Nevertheless, with careful selections in each of these steps, the quality (accuracy and precision) of the recorded data can be considered comparable

Assessing the relationship between subjective trust, confidence measurements, and mouse trajectory characteristics in an online task

Trust is essential for our interactions with others but also with artificial intelligence (AI) based systems. To understand whether a user trusts an AI, researchers need reliable measurement tools. However, currently discussed markers mostly rely on expensive and invasive sensors, like electroencephalograms, which may cause discomfort. The analysis of mouse trajectory has been suggested as a convenient tool for trust assessment. However, the relationship between trust, confidence and mouse trajectory is not yet fully understood. To provide more insights into this relationship, we asked participants (n = 146) to rate whether several tweets were offensive while an AI suggested its assessment. Our results reveal which aspects of the mouse trajectory are affected by the users subjective trust and confidence ratings; yet they indicate that these measures might not explain sufficiently the variance to be used on their own. This work examines a potential low-cost trust assessment in AI systems.Comment: Submitted to CHI 2023 and rejecte

TASK HANDOFF BETWEEN HUMANS AND AUTOMATION

The Department of Defense (DOD) seeks to incorporate human-automation teaming to decrease human operators’ cognitive workload, especially in the context of future vertical lift (FVL). Researchers created a “wizard of oz” study to observe human behavior changes as task difficulty and levels of automation increased. The platform used for the study was a firefighting strategy software game called C3Fire. Participants were paired with a confederate acting as an automated agent to observe the participant’s behavior in a human-automation team. The independent variables were automation level (within; low, medium, high) and queuing (between; uncued, cued). The dependent variables were the number of messages transmitted to the confederate, the number of tasks embedded in those messages (tasks handed off), and the participant’s self-reported cognitive workload score. The study results indicated that as the confederate increased its scripted level of automation, the number of tasks handed off to automation increased. However, the number of messages transmitted to automation and the subjective cognitive workload remained the same. The study’s findings suggest that while human operators were able to bundle tasks, cognitive workload remained relatively unchanged. The results imply that the automation level may have less impact on cognitive workload than anticipated.Major, United States ArmyCaptain, United States ArmyCaptain, United States ArmyCaptain, United States ArmyCaptain, United States ArmyApproved for public release. Distribution is unlimited

Old habits die hard? The fragility of eco-driving mental models and why green driving behaviour is difficult to sustain

Tangible incentives, training and feedback systems have been shown to reduce drivers’ fuel consumption in several studies. However, the effects of such tools are often short-lived or dependent on continuous cues. Several studies found that many drivers already possess eco-driving mental models, and are able to activate them, for instance when an experimenter asks them to “drive fuel-efficiently”. However, it is unclear how sustainable mental models are. The aim of the current study was to investigate the resilience of drivers’ eco-driving mental models following engagement with a workload task, implemented as a simplified version of the Twenty Questions Task (TQT). Would drivers revert to ‘everyday’ driving behaviours following exposure to heightened workload? A driving simulator experiment was conducted whereby 15 participants first performed a baseline drive, and then in a second session were prompted to drive fuel-efficiently. In each drive, the participants drove with and without completing the TQT. The results of two-way ANOVAs and Wilcoxon signed-rank tests support that they drive more slowly and keep a more stable speed when asked to eco-drive. However, it appears that drivers fell back into ‘everyday’ habits over time, and after the workload task, but these effects cannot be clearly isolated from each other. Driving and the workload task possibly invoked unrelated thoughts, causing eco-driving mental models to be deactivated. Future research is needed to explore ways to activate existing knowledge and skills and to use reminders at regular intervals, so new driver behaviours can be proceduralised and automatised and thus changed sustainably

Using eye tracking to account for attribute non-attendance in choice experiments

This study uses eye-tracking measures to account for attribute non-attendance (ANA) in choice experiments. Using the case of sustainability labelling on coffee, we demonstrate various approaches to account for ANA based on number of fixation count cut-offs, definitions for detecting ignored attributes, and methods for modelling ANA. Some of the sustainability attributes identified through eye-tracking measures as being ‘visually ignored’ were truly ignored while in none of the tested approaches price was truly ignored. The adequacy of eye-tracking as a visual ANA measure might thus depend on the attribute. Further, there are inconsistencies in identifying non-attenders using visual ANA and the coefficient of variation. Based on our results, we cannot conclude that eye-tracking always adequately identifies ANA. However, we identified several major challenges that can assist in further optimizing the use of eye-tracking in the context of ANA

Proceedings of the 2nd IUI Workshop on Interacting with Smart Objects

These are the Proceedings of the 2nd IUI Workshop on Interacting with Smart Objects. Objects that we use in our everyday life are expanding their restricted interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are thus able to capture information, process and store it and interact with their environments, turning them into smart objects

Détection et amélioration de l'état cognitif de l'apprenant

Cette thèse vise à détecter et améliorer l’état cognitif de l’apprenant. Cet état est défini par la capacité d’acquérir de nouvelles connaissances et de les stocker dans la mémoire. Nous nous sommes essentiellement intéressés à améliorer le raisonnement des apprenants, et ceci dans trois environnements : environnement purement cognitif Logique, jeu sérieux LewiSpace et jeu sérieux intelligent Inertia. La détection de cet état se fait essentiellement par des mesures physiologiques (en particulier les électroencéphalogrammes) afin d’avoir une idée sur les interactions des apprenants et l’évolution de leurs états mentaux. L’amélioration des performances des apprenants et de leur raisonnement est une clé pour la réussite de l’apprentissage. Dans une première partie, nous présentons l’implémentation de l’environnement cognitif logique. Nous décrivons des statistiques faites sur cet environnement. Nous avons collecté durant une étude expérimentale les données sur l’engagement, la charge cognitive et la distraction. Ces trois mesures se sont montrées efficaces pour la classification et la prédiction des performances des apprenants. Dans une deuxième partie, nous décrivons le jeu Lewispace pour l’apprentissage des diagrammes de Lewis. Nous avons mené une étude expérimentale et collecté les données des électroencéphalogrammes, des émotions et des traceurs de regard. Nous avons montré qu’il est possible de prédire le besoin d’aide dans cet environnement grâce à ces mesures physiologiques et des algorithmes d’apprentissage machine. Dans une troisième partie, nous clôturons la thèse en présentant des stratégies d’aide intégrées dans un jeu virtuel Inertia (jeu de physique). Cette dernière s’adapte selon deux mesures extraites des électroencéphalogrammes (l’engagement et la frustration). Nous avons montré que ce jeu permet d’augmenter le taux de réussite dans ses missions, la performance globale et par conséquent améliorer l’état cognitif de l’apprenant.This thesis aims at detecting and enhancing the cognitive state of a learner. This state is measured by the ability to acquire new knowledge and store it in memory. Focusing on three types of environments to enhance reasoning: environment Logic, serious game LewiSpace and intelligent serious game Inertia. Physiological measures (in particular the electroencephalograms) have been taken in order to measure learners’ engagement and mental states. Improving learners’ reasoning is key for successful learning process. In a first part, we present the implementation of logic environment. We present statistics on this environment, with data collected during an experimental study. Three types of data: engagement, workload and distraction, these measures were effective and can predict and classify learner’s performance. In a second part, we describe the LewiSpace game, aimed at teaching Lewis diagrams. We conducted an experimental study and collected data from electroencephalograms, emotions and eye-tracking software. Combined with machine learning algorithms, it is possible to anticipate a learner’s need for help using these data. In a third part, we finish by presenting some assistance strategies in a virtual reality game called Inertia (to teach Physics). The latter adapts according to two measures extracted from electroencephalograms (frustration and engagement). Based on our study, we were able to enhance the learner’s success rate on game missions, by improving its cognitive state

Learning by generative drawing: An analysis of learning behavior and eye movements to evaluate theoretical assumptions

Das Lernen mit komplexen Sachtexten stellt für viele Schülerinnen und Schüler ein großes Hindernis dar. Beim verstehenden Lesen eines solchen Textes haben Lernende häufig Schwierigkeiten, verschiedene Teile des Textes zu integrieren, ein genaues Textverständnis zu entwickeln und spezifisches Wissen gezielt zu nutzen, um das Gelesene zu interpretieren (Bos, Valtin, Hußmann, Wendt & Goy, 2017; Hußmann et al., 2017; Weis, Zehner, Sälzer, Strohmaier & Pfost, 2016). Das Hinzufügen von Abbildungen zu einem Text kann zwar das fachliche Verstehen maßgeblich unterstützen (Mayer, 2009, 2014c), jedoch gibt es auch Hinweise darauf, dass Lernende vorgegebene Abbildungen häufig nur oberflächlich betrachten und verstehen (z. B. Brandstetter-Korinth, 2017; Cook, Carter & Wiebe, 2008; Hannus & Hyönä, 1999). Eine Alternative bietet die Lernstrategie des sinnstiftenden Zeichnens, bei welcher Lernende selbst repräsentationale Abbildungen zu den zentralen Sachverhalten eines Textes erstellen (Alesandrini, 1984; Carney & Levin, 2002; van Meter & Garner, 2005). Im Gegensatz zum Lernen mit einem Text und vorgegebenen Abbildungen verlassen Lernende bei der Anwendung der Lernstrategie des sinnstiftenden Zeichnens die passive Rolle eines Rezipienten von Text- und Bildmaterial und gestalten stattdessen den Lernprozess aktiv mit, indem sie ihre Aufmerksamkeit gezielt den Schlüsselstellen eines Textes zuwenden, die relevanten Informationen aus dem Text selektieren, in mentalen Repräsentationen organisieren und diese schließlich mit dem Vorwissen in ein kohärentes mentales Modell integrieren (van Meter & Garner, 2005; van Meter & Firetto, 2013). Da die Lernenden die Zeichnungen von Grund auf Stück für Stück selbst zusammensetzen und sich so intensiv mit den verschiedenen Ebenen sowie einzelnen Elementen der Abbildung und derer Zusammenhänge auseinandersetzen, erhalten sie zudem über den Visualisierungsprozess einen intuitiven Zugang, um Bildinformationen aus komplexen Abbildunggen zu entnehmen. Die Wirksamkeit des sinnstiftenden Zeichnens wird im zugrundeliegenden Cognitive Model of Drawing Construction (van Meter & Firetto, 2013) darauf zurückgeführt, dass Lernende beim Prozess der Bildgenerierung einen dreiphasigen Selbstregulationskreislauf durchlaufen, welcher die Ausführung der kognitiven Prozesse steuert. Weiterhin werden durch den Selbstregulationskreislauf metakognitive Prozesse der Selbstüberwachung und -regulation angestoßen, welche dafür verantwortlich sind, dass die Aufmerksamkeit der Lernenden verstärkt auf die zentralen Stellen des zu bearbeitenden Textes gelenkt wird und sinnstiftende Selektions-, Organisations- und Integrationsprozesse stattfinden, sodass unter Einbezug des Vorwissens ein kohärentes mentales Modell konstruiert werden kann. Durch das Erstellen einer Zeichnung erhalten die Lernenden außerdem eine direkte Rückmeldung darüber, ob die wesentlichen Aspekte des Textes erfasst und verstanden worden sind oder ob sich erneut mit dem Text auseinandergesetzt werden muss. Bisherige Befunde zum sinnstiftenden Zeichnens legen nahe, dass die Lernstrategie dann ihr volles Potenzial entfalten kann, wenn Lernende qualitativ hochwertige Zeichnungen erstellen können, ohne sich beim Visualisierungsprozess kognitiv zu überlasten (Schmeck, 2010). Unter diesen Voraussetzungen kann das sinnstiftende Zeichnen sowohl das Textverständnis als auch die Transferleistungen fördern (für eine Übersicht siehe Fiorella & Mayer, 2015; Leutner & Schmeck, 2014; van Meter & Firetto, 2013). Weiterhin erweist sich die Qualität der von den Lernenden erstellten Zeichnungen als geeigneter Prädiktor für den Lernerfolg (prognostic drawing principle; Leutner & Schmeck, 2014; Schwamborn, Mayer, Thillmann, Leopold & Leutner, 2010). Während die Lernförderlichkeit der Lernstrategie des sinnstiftenden Zeichnens empirisch gut belegt ist, gibt es bisher jedoch keine empirischen Belege für die in den theoretischen Modellen zum sinnstiftenden Zeichnen angenommenen zugrundeliegenden kognitiven und metakognitiven Prozesse. Anhand der Analyse von Blickbewegungen und Verhaltensspuren sind in der vorliegenden Arbeit Indikatoren für kognitive Verarbeitungsprozesse während des sinnstiftenden Zeichnens ausgemacht worden, sodass die theoretischen Annahmen im Cognitive Model of Drawing Construction (van Meter & Firetto, 2013) hinsichtlich des Selbstregulationskreislaufs und des Einflusses des Vorwissens auf die kognitiven Verarbeitungsprozesse empirisch überprüft werden konnten. In der in dieser Arbeit präsentierten ersten und zweiten Studie lag der Fokus daher auf der Frage, welche Blickbewegungsmuster sich bei der Anwendung der Lernstrategie des sinnstiftenden Zeichnens nachweisen lassen und inwiefern sich diese Blickbewegungsmuster als Indikatoren für kognitive Verarbeitungsprozesse der Lernenden von denen solcher Lernenden unterscheiden, die vorgegebene Abbildungen zu einem Text erhalten (Studie I) oder die Lernstrategie des Zusammenfassens ausführen (Studie II). Weiterhin wurde in beiden Studien untersucht, inwiefern sich die Lernenden im Hinblick auf den Lernerfolg unterscheiden und ob bei Lernenden, welche sinnstiftende Zeichnungen erstellen, die Qualität der Zeichnungen prädiktiv für den Lernerfolg ist. Die Ergebnisse der Blickbewegungsanalysen und Lernerfolgstests legen nahe, dass, im Vergleich zu einer klassischen multimedialen Lernumgebung oder der Anwendung der Lernstrategie des Zusammenfassens, sinnstiftendes Zeichnen zu einer strategisch fokussierteren Nutzung der kognitiven Prozesse des Selektierens und Integrierens führt. Die gefundenen Blickbewegungsmuster für die Lernstrategie des sinnstiftenden Zeichnens stehen dabei im Einklang mit den Annahmen des theoretischen Modells: Lernende, welche sinnstiftende Zeichnungen zu einem Text erstellten, setzten sich wiederholt intensiv mit dem Text und den sich in der Entstehung befindenden Abbildungen auseinander und richteten dabei ihre Aufmerksamkeit verstärkt auf die zentralen Stellen des Textes, sodass ein höherer Anteil an sinnstiftenden Verknüpfungen zwischen depiktiven und deskriptiven Informationen erzeugt werden konnte. Weiterhin erwies sich die Qualität der während des Lernens erstellen Zeichnungen als prädiktiv für den Lernerfolg. In der dritten Studie dieser Arbeit wurde schließlich der Einfluss des Vorwissens auf die kognitiven Verarbeitungsprozesse während des sinnstiftenden Zeichnens untersucht. Dabei konnte erwartungsgemäß gezeigt werden, dass das Vorwissen einen entscheidenden Einfluss auf die kognitiven Verarbeitungsprozesse während des sinnstiftenden Zeichnens ausübt und damit auch einen entscheidenden Einfluss auf die Konstruktion eines kohärenten mentalen Modells nimmt. Die Auswertung der Blickbewegungsmuster und Verhaltensspuren von Lernenden mit hohem und geringem Vorwissen bei der Strategieanwendung zeigen erwartungskonform, dass Lernende mit hohem Vorwissen nicht nur qualitativ hochwertigere Zeichnungen während des Lernens erstellen konnten und ein höheres Textverständnis und Transferwissen erwarben als Lernende mit geringem Vorwissen, sondern außerdem ausgeprägtere Selektionsprozesse aufwiesen. Im Hinblick auf sinnstiftende Integrationsprozesse konnte jedoch kein Unterschied zwischen Lernenden mit hohem und geringem Vorwissen bei der Anwendung der Lernstrategie des sinnstiftenden Zeichnens festgestellt werden. Die Qualität der Zeichnungen war wiederum positiv mit dem Lernerfolg verbunden. Insgesamt weisen die empirischen Erkenntnisse der vorliegenden Arbeit auf die Gültigkeit der im Cognitive Model of Drawing Construction (van Meter & Firetto, 2013) getroffenen Annahmen hinsichtlich des Selbstregulationskreislaufs und des Einflusses des Vorwissens auf die kognitiven Verarbeitungsprozesse hin. Weiterhin konnte gezeigt werden, dass Lernende besonders dann im Hinblick auf ihr Textverständnis vom Einsatz der Lernstrategie des sinnstiftenden Zeichnens profitieren, wenn sie in der Lage sind, qualitativ hochwertige Zeichnungen zu erstellen. Zudem erwies sich sinnstiftendes Zeichnen für Lernende mit geringem Vorwissen als sinnvoll, um ihren Wissensrückstand im Vergleich zu Strategieanwendenden mit hohem Vorwissen auszugleichen. Damit bietet sich der Einsatz der Lernstrategie des sinnstiftenden Zeichnens im schulischen Kontext insbesondere bei leistungsheterogen Klassen an, um komplexe Sachtexte sinnstiftend zu erarbeiten.The requirements of a complex scientific text can be a major obstacle for students who study on their own for deep level understanding. In the process of making sense of a text, learners often struggle to integrate different parts of the text, to develop a precise text comprehension, and to use specific textual knowledge to interpret what they have just read (Bos, Valtin, Hußmann, Wendt, & Goy, 2017; Hußmann et al., 2017; Weis, Zehner, Sälzer, Strohmaier, & Pfost, 2016). Although providing pictures in addition to a text can promote learning (Mayer, 2009, 2014c), students tend to look at pictures only in a superficial way and often have difficulties interpreting them (e.g., Brandstetter-Korinth, 2017; Cook, Carter, & Wiebe, 2008; Hannus & Hyönä, 1999). A promising approach to improve learning in this regard is to encourage students to draw their own representational pictures, which reflect the main ideas of the text (Alesandrini, 1984; Carney & Levin, 2002; van Meter & Garner, 2005). Contrary to learning with author-provided pictures that are just added to a text, learners who engage in generative drawing are no longer passive consumers of information, but are actively involved in generative processing such as selecting key elements and relations, organizing them into mental representations, and integrating the mental representations with each other and with prior knowledge into a coherent mental model (van Meter & Firetto, 2013; van Meter & Garner, 2005). Since learners deal with the individual elements of the picture and their relations more intensively and create their drawings piece by piece, they also reach a deeper level of understanding in how to deal with complex pictures and how to extract information from these pictures. The cognitive and metacognitive processes underlying generative drawing are described on a theoretical basis in the Cognitive Model of Drawing Construction (van Meter & Firetto, 2013), in which the processes of selecting, organizing, and integrating are interpreted in terms of self-regulated learning. When learners engage in generative drawing, they undergo a self-regulation cycle that begins with setting performance standards for the drawing by deciding on how many details need to be included and how to express relations between different parts. Metacognitive processes of self-monitoring and self-regulation are triggered, when learners compare their in-progress work to the standards set earlier. If learners are unable to reach the standards or have difficulties to externalize the mental model, metacognitive control guides them back to the instructional material to re-engage in the cognitive processes of selecting, organizing, and integrating in order to revise their mental model. Thus, van Meter and Firetto (2013) predict that, by using the drawing strategy, learners’ attention is directed towards key elements and relations in the text and that learners who generate drawings on their own use self-monitoring and self-regulation processes more frequently than learners who do not use this strategy. Research shows that generative drawing is more likely to develop its full potential when the drawing process itself is supported. Providing a legend showing all relevant elements for drawing, for example, can reduce extraneous cognitive processing that the mechanics of drawing itself induce (Schmeck, 2010). Under this boundary condition, generative drawing as a self-regulated learning strategy can foster deep level understanding that leads to better learning outcomes in retention and transfer tests (for an overview see Fiorella & Mayer, 2015; Leutner & Schmeck, 2014; van Meter & Firetto, 2013). Moreover, the quality of learners’ drawings during learning predicts the quality of their learning outcomes (prognostic drawing principle; Leutner & Schmeck, 2014; Schwamborn, Mayer, Thillmann, Leopold, & Leutner, 2010). While there is strong evidence that generative drawing promotes a deeper understanding of the learning materials, there is a lack of empirical evidence for the proposed underlying cognitive and metacognitive processes. To shed more light on the theoretical assumptions made in the Cognitive Model of Drawing Construction (van Meter & Firetto, 2013) concerning the self-regulation cycle and the influence of prior knowledge on cognitive processing, three studies were conducted, in which students' learning processes were analyzed as they engaged in generative drawing using eye-tracking measures and students' learning outcomes using posttest measures. The purpose of the first and second study presented in this thesis was to examine if students exhibit different eye-movement patterns as indicators of cognitive processing during learning when they generate drawings than when they are given author-generated pictures in addition to a text (Study I) or when they generate written summaries (Study II). Furthermore, in both studies was examined how learners differed in learning outcome performance. A secondary goal was to determine whether the quality of the drawings was predictive for the quality of learning outcomes. The results of both experiments show that learners who engaged in generative drawing during reading a scientific text displayed more strategically focused processing of the text by focusing more attention on relevant text passages and connections between generated drawings and relevant text passages than learners who received a different instructional strategy (such as providing pictures in Study I) or who were prompted to use a different generative learning strategy (such as writing summaries in Study II). Thus, the results are in line with the assumptions made in the Cognitive Model of Drawing Construction (van Meter & Firetto, 2013): Learners who engage in generative drawing are more likely to direct their attention towards key elements and their relations in the text and to engage in meaningful self-monitoring and self-regulation processes in order to externalize the drawing. Moreover, the quality of the drawings was positively associated with the quality of learning outcomes. The purpose of the third study presented in this thesis was to investigate the impact of prior knowledge on generative processing that leads to mental model construction during generative drawing. As expected, learners with high prior knowledge not only created drawings of significantly higher quality during learning and scored higher on all learning outcome measures than learners with low prior knowledge, but they also were better able to distinguish between important and less important information in the text, indicating more profound selection processing. However, both learners with high and low prior knowledge did not differ in making meaningful connections between their drawings and corresponding text passages. Furthermore, the quality of the drawings proved to be predictive of the quality of learning outcomes. Overall, the empirical findings of this thesis contribute to evaluating the theoretical assumptions of the Cognitive Model of Drawing Construction (van Meter & Firetto, 2013) with regard to the underlying cognitive and metacognitive processes and the impact of prior knowledge on mental model construction. Moreover, learners benefit most from using the drawing strategy, when they are able to produce high-quality drawings. In particular, generative drawing is a strategy that is suitable for low-prior-knowledge learners in order to catch up with the knowledge that learners with high prior knowledge already possess before learning to a certain extent. Thus, generative drawing as a self-regulated learning strategy should be explicitly used in performance-heterogeneous classes at school in order to help students deal with complex scientific texts

Modeling Learner Mood In Realtime Through Biosensors For Intelligent Tutoring Improvements

Computer-based instructors, just like their human counterparts, should monitor the emotional and cognitive states of their students in order to adapt instructional technique. Doing so requires a model of student state to be available at run time, but this has historically been difficult. Because people are different, generalized models have not been able to be validated. As a person’s cognitive and affective state vary over time of day and seasonally, individualized models have had differing difficulties. The simultaneous creation and execution of an individualized model, in real time, represents the last option for modeling such cognitive and affective states. This dissertation presents and evaluates four differing techniques for the creation of cognitive and affective models that are created on-line and in real time for each individual user as alternatives to generalized models. Each of these techniques involves making predictions and modifications to the model in real time, addressing the real time datastream problems of infinite length, detection of new concepts, and responding to how concepts change over time. Additionally, with the knowledge that a user is physically present, this work investigates the contribution that the occasional direct user query can add to the overall quality of such models. The research described in this dissertation finds that the creation of a reasonable quality affective model is possible with an infinitesimal amount of time and without “ground truth” knowledge of the user, which is shown across three different emotional states. Creation of a cognitive model in the same fashion, however, was not possible via direct AI modeling, even with all of the “ground truth” information available, which is shown across four different cognitive states