Latent gaze information in highly dynamic decision-tasks

Die Digitalisierung durchdringt immer mehr Lebensbereiche. Aufgaben werden zunehmend digital erledigt und damit schneller, effizienter, aber auch zielorientierter und erfolgreicher erfüllt. Die rasante Entwicklung im Bereich der künstlichen Intelligenz in den letzten Jahren hat dabei eine große Rolle gespielt, denn sie hat viele hilfreiche Ansätze hervorgebracht, auf die immer weiter aufgebaut werden kann. Gleichzeitig werden die Augen, ihre Bewegungen und die Bedeutung dieser Bewegungen immer weiter erforscht. Die Verknüpfung dieser Entwicklungen hat zu spannenden Ansätzen in der Wissenschaft geführt. In dieser Dissertation stelle ich einige der Ansätze vor, an denen ich während meiner Promotion gearbeitet habe. Zunächst gebe ich einen Einblick in die Entwicklung von Modellen, die mit Hilfe künstlicher Intelligenz Verbindungen zwischen Augenbewegungsdaten und visueller Expertise herstellen. Dies wird anhand zwei verschiedener Bereiche, genauer gesagt zwei verschiedener Personengruppen, demonstriert: Sportler bei Entscheidungsfindungen und Chirurgen bei arthroskopischen Eingriffen. Die daraus resultierenden Modelle können als digitale Diagnosemodelle für die automatische Erkennung von visueller Expertise betrachtet werden. Darüber hinaus stelle ich Ansätze vor, die die Übertragbarkeit von Augenbewegungsmustern auf verschiedene Kompetenzbereiche untersuchen sowie wichtige Aspekte von Techniken zur Generalisierung. Schließlich befasse ich mich mit der zeitlichen Erkennung von Verwirrung auf der Grundlage von Augenbewegungsdaten. Die Ergebnisse legen eine Nutzung der Modelle als Zeitgeber für mögliche digitale Assistenzoptionen in der Ausbildung von Berufsanfängern nahe. Eine Besonderheit meiner Untersuchungen besteht darin, dass ich auf sehr wervolle Daten von DFB-Jugendkaderathleten sowie von langjährigen Experten in der Arthroskopie zurückgreifen konnte. Insbesondere die Arbeit mit den DFB-Daten stieß auf das Interesse von Radiound Printmedien, genauer, DeutschlandFunk Nova und SWR DasDing. Alle hier vorgestellten Beiträge wurden in international renommierten Fachzeitschriften oder auf Konferenzen veröffentlicht.Digitization is penetrating more and more areas of life. Tasks are increasingly being completed digitally, and are therefore not only fulfilled faster, more efficiently but also more purposefully and successfully. The rapid developments in the field of artificial intelligence in recent years have played a major role in this, as they brought up many helpful approaches to build on. At the same time, the eyes, their movements, and the meaning of these movements are being progressively researched. The combination of these developments has led to exciting approaches. In this dissertation, I present some of these approaches which I worked on during my Ph.D. First, I provide insight into the development of models that use artificial intelligence to connect eye movements with visual expertise. This is demonstrated for two domains or rather groups of people: athletes in decision-making actions and surgeons in arthroscopic procedures. The resulting models can be considered as digital diagnostic models for automatic expertise recognition. Furthermore, I show approaches that investigate the transferability of eye movement patterns to different expertise domains and subsequently, important aspects of techniques for generalization. Finally, I address the temporal detection of confusion based on eye movement data. The results suggest the use of the resulting model as a clock signal for possible digital assistance options in the training of young professionals. An interesting aspect of my research is that I was able to draw on very valuable data from DFB youth elite athletes as well as on long-standing experts in arthroscopy. In particular, the work with the DFB data attracted the interest of radio and print media, namely DeutschlandFunk Nova and SWR DasDing. All resulting articles presented here have been published in internationally renowned journals or at conferences

VisionaryVR: An Optical Simulation Tool for Evaluating and Optimizing Vision Correction Solutions in Virtual Reality

Developing and evaluating vision science methods require robust and efficient tools for assessing their performance in various real-world scenarios. This study presents a novel virtual reality (VR) simulation tool that simulates real-world optical methods while giving high experimental control to the experiment. The tool incorporates an experiment controller, to smoothly and easily handle multiple conditions, a generic eye-tracking controller, that works with most common VR eye-trackers, a configurable defocus simulator, and a generic VR questionnaire loader to assess participants' behavior in virtual reality. This VR-based simulation tool bridges the gap between theoretical and applied research on new optical methods, corrections, and therapies. It enables vision scientists to increase their research tools with a robust, realistic, and fast research environment

How to Tune Autofocals: A Comparative Study of Advanced Tuning Methods

This study comprehensively evaluates tuning methods for autofocal glasses using virtual reality (VR), addressing the challenge of presbyopia. With aging, presbyopia diminishes the eye's ability to focus on nearby objects, impacting the quality of life for billions. Autofocals, employing focus-tunable lenses, dynamically adjust optical power for each fixation, promising a more natural visual experience than traditional bifocal or multifocal lenses. Our research contrasts the most common tuning methods - manual, gaze-based, and vergence - within a VR setup to mimic real-world scenarios. Utilizing the XTAL VR headset equipped with eye-tracking, the study replicated autofocal scenarios, measuring performance and usability through psychophysical tasks and NASA TLX surveys. Results show varying strengths and weaknesses across methods, with gaze control excelling in precision but not necessarily comfort and manual control providing stability and predictability. The findings guide the selection of tuning methods based on task requirements and user preferences, highlighting a balance between precision and ease of use

Assessing the relationship between subjective trust, confidence measurements, and mouse trajectory characteristics in an online task

Trust is essential for our interactions with others but also with artificial intelligence (AI) based systems. To understand whether a user trusts an AI, researchers need reliable measurement tools. However, currently discussed markers mostly rely on expensive and invasive sensors, like electroencephalograms, which may cause discomfort. The analysis of mouse trajectory has been suggested as a convenient tool for trust assessment. However, the relationship between trust, confidence and mouse trajectory is not yet fully understood. To provide more insights into this relationship, we asked participants (n = 146) to rate whether several tweets were offensive while an AI suggested its assessment. Our results reveal which aspects of the mouse trajectory are affected by the users subjective trust and confidence ratings; yet they indicate that these measures might not explain sufficiently the variance to be used on their own. This work examines a potential low-cost trust assessment in AI systems.Comment: Submitted to CHI 2023 and rejecte

Latent gaze information in highly dynamic decision-tasks

Die Digitalisierung durchdringt immer mehr Lebensbereiche. Aufgaben werden zunehmend digital erledigt und damit schneller, effizienter, aber auch zielorientierter und erfolgreicher erfüllt. Die rasante Entwicklung im Bereich der künstlichen Intelligenz in den letzten Jahren hat dabei eine große Rolle gespielt, denn sie hat viele hilfreiche Ansätze hervorgebracht, auf die immer weiter aufgebaut werden kann. Gleichzeitig werden die Augen, ihre Bewegungen und die Bedeutung dieser Bewegungen immer weiter erforscht. Die Verknüpfung dieser Entwicklungen hat zu spannenden Ansätzen in der Wissenschaft geführt. In dieser Dissertation stelle ich einige der Ansätze vor, an denen ich während meiner Promotion gearbeitet habe. Zunächst gebe ich einen Einblick in die Entwicklung von Modellen, die mit Hilfe künstlicher Intelligenz Verbindungen zwischen Augenbewegungsdaten und visueller Expertise herstellen. Dies wird anhand zwei verschiedener Bereiche, genauer gesagt zwei verschiedener Personengruppen, demonstriert: Sportler bei Entscheidungsfindungen und Chirurgen bei arthroskopischen Eingriffen. Die daraus resultierenden Modelle können als digitale Diagnosemodelle für die automatische Erkennung von visueller Expertise betrachtet werden. Darüber hinaus stelle ich Ansätze vor, die die Übertragbarkeit von Augenbewegungsmustern auf verschiedene Kompetenzbereiche untersuchen sowie wichtige Aspekte von Techniken zur Generalisierung. Schließlich befasse ich mich mit der zeitlichen Erkennung von Verwirrung auf der Grundlage von Augenbewegungsdaten. Die Ergebnisse legen eine Nutzung der Modelle als Zeitgeber für mögliche digitale Assistenzoptionen in der Ausbildung von Berufsanfängern nahe. Eine Besonderheit meiner Untersuchungen besteht darin, dass ich auf sehr wervolle Daten von DFB-Jugendkaderathleten sowie von langjährigen Experten in der Arthroskopie zurückgreifen konnte. Insbesondere die Arbeit mit den DFB-Daten stieß auf das Interesse von Radiound Printmedien, genauer, DeutschlandFunk Nova und SWR DasDing. Alle hier vorgestellten Beiträge wurden in international renommierten Fachzeitschriften oder auf Konferenzen veröffentlicht.Digitization is penetrating more and more areas of life. Tasks are increasingly being completed digitally, and are therefore not only fulfilled faster, more efficiently but also more purposefully and successfully. The rapid developments in the field of artificial intelligence in recent years have played a major role in this, as they brought up many helpful approaches to build on. At the same time, the eyes, their movements, and the meaning of these movements are being progressively researched. The combination of these developments has led to exciting approaches. In this dissertation, I present some of these approaches which I worked on during my Ph.D. First, I provide insight into the development of models that use artificial intelligence to connect eye movements with visual expertise. This is demonstrated for two domains or rather groups of people: athletes in decision-making actions and surgeons in arthroscopic procedures. The resulting models can be considered as digital diagnostic models for automatic expertise recognition. Furthermore, I show approaches that investigate the transferability of eye movement patterns to different expertise domains and subsequently, important aspects of techniques for generalization. Finally, I address the temporal detection of confusion based on eye movement data. The results suggest the use of the resulting model as a clock signal for possible digital assistance options in the training of young professionals. An interesting aspect of my research is that I was able to draw on very valuable data from DFB youth elite athletes as well as on long-standing experts in arthroscopy. In particular, the work with the DFB data attracted the interest of radio and print media, namely DeutschlandFunk Nova and SWR DasDing. All resulting articles presented here have been published in internationally renowned journals or at conferences

Expertise classification of soccer goalkeepers in highly dynamic decision tasks : a deep learning approach for temporal and spatial feature recognition of fixation image patch sequences

The focus of expertise research moves constantly forward and includes cognitive factors, such as visual information perception and processing. In highly dynamic tasks, such as decision making in sports, these factors become more important to build a foundation for diagnostic systems and adaptive learning environments. Although most recent research focuses on behavioral features, the underlying cognitive mechanisms have been poorly understood, mainly due to a lack of adequate methods for the analysis of complex eye tracking data that goes beyond aggregated fixations and saccades. There are no consistent statements about specific perceptual features that explain expertise. However, these mechanisms are an important part of expertise, especially in decision making in sports games, as highly trained perceptual cognitive abilities can provide athletes with some advantage. We developed a deep learning approach that independently finds latent perceptual features in fixation image patches. It then derives expertise based solely on these fixation patches, which encompass the gaze behavior of athletes in an elaborately implemented virtual reality setup. We present a CNN-BiLSTM based model for expertise assessment in goalkeeper-specific decision tasks on initiating passes in build-up situations. The empirical validation demonstrated that our model has the ability to find valuable latent features that detect the expertise level of 33 athletes (novice, advanced, and expert) with 73.11% accuracy. This model is a first step in the direction of generalizable expertise recognition based on eye movements