Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

An enactive approach to perceptual augmentation in mobility

Event predictions are an important constituent of situation awareness, which is a key objective for many applications in human-machine interaction, in particular in driver assistance. This work focuses on facilitating event predictions in dynamic environments. Its primary contributions are 1) the theoretical development of an approach for enabling people to expand their sampling and understanding of spatiotemporal information, 2) the introduction of exemplary systems that are guided by this approach, 3) the empirical investigation of effects functional prototypes of these systems have on human behavior and safety in a range of simulated road traffic scenarios, and 4) a connection of the investigated approach to work on cooperative human-machine systems. More specific contents of this work are summarized as follows: The first part introduces several challenges for the formation of situation awareness as a requirement for safe traffic participation. It reviews existing work on these challenges in the domain of driver assistance, resulting in an identification of the need to better inform drivers about dynamically changing aspects of a scene, including event probabilities, spatial and temporal distances, as well as a suggestion to expand the scope of assistance systems to start informing drivers about relevant scene elements at an early stage. Novel forms of assistance can be guided by different fundamental approaches that target either replacement, distribution, or augmentation of driver competencies. A subsequent differentiation of these approaches concludes that an augmentation-guided paradigm, characterized by an integration of machine capabilities into human feedback loops, can be advantageous for tasks that rely on active user engagement, the preservation of awareness and competence, and the minimization of complexity in human- machine interaction. Consequently, findings and theories about human sensorimotor processes are connected to develop an enactive approach that is consistent with an augmentation perspective on human-machine interaction. The approach is characterized by enabling drivers to exercise new sensorimotor processes through which safety-relevant spatiotemporal information may be sampled. In the second part of this work, a concept and functional prototype for augmenting the perception of traffic dynamics is introduced as a first example for applying principles of this enactive approach. As a loose expression of functional biomimicry, the prototype utilizes a tactile inter- face that communicates temporal distances to potential hazards continuously through stimulus intensity. In a driving simulator study, participants quickly gained an intuitive understanding of the assistance without instructions and demonstrated higher driving safety in safety-critical highway scenarios. But this study also raised new questions such as whether benefits are due to a continuous time-intensity encoding and whether utility generalizes to intersection scenarios or highway driving with low criticality events. Effects of an expanded assistance prototype with lane-independent risk assessment and an option for binary signaling were thus investigated in a separate driving simulator study. Subjective responses confirmed quick signal understanding and a perception of spatial and temporal stimulus characteristics. Surprisingly, even for a binary assistance variant with a constant intensity level, participants reported perceiving a danger-dependent variation in stimulus intensity. They further felt supported by the system in the driving task, especially in difficult situations. But in contrast to the first study, this support was not expressed by changes in driving safety, suggesting that perceptual demands of the low criticality scenarios could be satisfied by existing driver capabilities. But what happens if such basic capabilities are impaired, e.g., due to poor visibility conditions or other situations that introduce perceptual uncertainty? In a third driving simulator study, the driver assistance was employed specifically in such ambiguous situations and produced substantial safety advantages over unassisted driving. Additionally, an assistance variant that adds an encoding of spatial uncertainty was investigated in these scenarios. Participants had no difficulties to understand and utilize this added signal dimension to improve safety. Despite being inherently less informative than spatially precise signals, users rated uncertainty-encoding signals as equally useful and satisfying. This appreciation for transparency of variable assistance reliability is a promising indicator for the feasibility of an adaptive trust calibration in human-machine interaction and marks one step towards a closer integration of driver and vehicle capabilities. A complementary step on the driver side would be to increase transparency about the driver’s mental states and thus allow for mutual adaptation. The final part of this work discusses how such prerequisites of cooperation may be achieved by monitoring mental state correlates observable in human behavior, especially in eye movements. Furthermore, the outlook for an addition of cooperative features also raises new questions about the bounds of identity as well as practical consequences of human-machine systems in which co-adapting agents may exercise sensorimotor processes through one another.Die Vorhersage von Ereignissen ist ein Bestandteil des Situationsbewusstseins, dessen Unterstützung ein wesentliches Ziel diverser Anwendungen im Bereich Mensch-Maschine Interaktion ist, insbesondere in der Fahrerassistenz. Diese Arbeit zeigt Möglichkeiten auf, Menschen bei Vorhersagen in dynamischen Situationen im Straßenverkehr zu unterstützen. Zentrale Beiträge der Arbeit sind 1) eine theoretische Auseinandersetzung mit der Aufgabe, die menschliche Wahrnehmung und das Verständnis von raum-zeitlichen Informationen im Straßenverkehr zu erweitern, 2) die Einführung beispielhafter Systeme, die aus dieser Betrachtung hervorgehen, 3) die empirische Untersuchung der Auswirkungen dieser Systeme auf das Nutzerverhalten und die Fahrsicherheit in simulierten Verkehrssituationen und 4) die Verknüpfung der untersuchten Ansätze mit Arbeiten an kooperativen Mensch-Maschine Systemen. Die Arbeit ist in drei Teile gegliedert: Der erste Teil stellt einige Herausforderungen bei der Bildung von Situationsbewusstsein vor, welches für die sichere Teilnahme am Straßenverkehr notwendig ist. Aus einem Vergleich dieses Überblicks mit früheren Arbeiten zeigt sich, dass eine Notwendigkeit besteht, Fahrer besser über dynamische Aspekte von Fahrsituationen zu informieren. Dies umfasst unter anderem Ereigniswahrscheinlichkeiten, räumliche und zeitliche Distanzen, sowie eine frühere Signalisierung relevanter Elemente in der Umgebung. Neue Formen der Assistenz können sich an verschiedenen grundlegenden Ansätzen der Mensch-Maschine Interaktion orientieren, die entweder auf einen Ersatz, eine Verteilung oder eine Erweiterung von Fahrerkompetenzen abzielen. Die Differenzierung dieser Ansätze legt den Schluss nahe, dass ein von Kompetenzerweiterung geleiteter Ansatz für die Bewältigung jener Aufgaben von Vorteil ist, bei denen aktiver Nutzereinsatz, die Erhaltung bestehender Kompetenzen und Situationsbewusstsein gefordert sind. Im Anschluss werden Erkenntnisse und Theorien über menschliche sensomotorische Prozesse verknüpft, um einen enaktiven Ansatz der Mensch-Maschine Interaktion zu entwickeln, der einer erweiterungsgeleiteten Perspektive Rechnung trägt. Dieser Ansatz soll es Fahrern ermöglichen, sicherheitsrelevante raum-zeitliche Informationen über neue sensomotorische Prozesse zu erfassen. Im zweiten Teil der Arbeit wird ein Konzept und funktioneller Prototyp zur Erweiterung der Wahrnehmung von Verkehrsdynamik als ein erstes Beispiel zur Anwendung der Prinzipien dieses enaktiven Ansatzes vorgestellt. Dieser Prototyp nutzt vibrotaktile Aktuatoren zur Kommunikation von Richtungen und zeitlichen Distanzen zu möglichen Gefahrenquellen über die Aktuatorposition und -intensität. Teilnehmer einer Fahrsimulationsstudie waren in der Lage, in kurzer Zeit ein intuitives Verständnis dieser Assistenz zu entwickeln, ohne vorher über die Funktionalität unterrichtet worden zu sein. Sie zeigten zudem ein erhöhtes Maß an Fahrsicherheit in kritischen Verkehrssituationen. Doch diese Studie wirft auch neue Fragen auf, beispielsweise, ob der Sicherheitsgewinn auf kontinuierliche Distanzkodierung zurückzuführen ist und ob ein Nutzen auch in weiteren Szenarien vorliegen würde, etwa bei Kreuzungen und weniger kritischem longitudinalen Verkehr. Um diesen Fragen nachzugehen, wurden Effekte eines erweiterten Prototypen mit spurunabhängiger Kollisionsprädiktion, sowie einer Option zur binären Kommunikation möglicher Kollisionsrichtungen in einer weiteren Fahrsimulatorstudie untersucht. Auch in dieser Studie bestätigen die subjektiven Bewertungen ein schnelles Verständnis der Signale und eine Wahrnehmung räumlicher und zeitlicher Signalkomponenten. Überraschenderweise berichteten Teilnehmer größtenteils auch nach der Nutzung einer binären Assistenzvariante, dass sie eine gefahrabhängige Variation in der Intensität von taktilen Stimuli wahrgenommen hätten. Die Teilnehmer fühlten sich mit beiden Varianten in der Fahraufgabe unterstützt, besonders in Situationen, die von ihnen als kritisch eingeschätzt wurden. Im Gegensatz zur ersten Studie hat sich diese gefühlte Unterstützung nur geringfügig in einer messbaren Sicherheitsveränderung widergespiegelt. Dieses Ergebnis deutet darauf hin, dass die Wahrnehmungsanforderungen der Szenarien mit geringer Kritikalität mit den vorhandenen Fahrerkapazitäten erfüllt werden konnten. Doch was passiert, wenn diese Fähigkeiten eingeschränkt werden, beispielsweise durch schlechte Sichtbedingungen oder Situationen mit erhöhter Ambiguität? In einer dritten Fahrsimulatorstudie wurde das Assistenzsystem in speziell solchen Situationen eingesetzt, was zu substantiellen Sicherheitsvorteilen gegenüber unassistiertem Fahren geführt hat. Zusätzlich zu der vorher eingeführten Form wurde eine neue Variante des Prototyps untersucht, welche räumliche Unsicherheiten der Fahrzeugwahrnehmung in taktilen Signalen kodiert. Studienteilnehmer hatten keine Schwierigkeiten, diese zusätzliche Signaldimension zu verstehen und die Information zur Verbesserung der Fahrsicherheit zu nutzen. Obwohl sie inherent weniger informativ sind als räumlich präzise Signale, bewerteten die Teilnehmer die Signale, die die Unsicherheit übermitteln, als ebenso nützlich und zufriedenstellend. Solch eine Wertschätzung für die Transparenz variabler Informationsreliabilität ist ein vielversprechendes Indiz für die Möglichkeit einer adaptiven Vertrauenskalibrierung in der Mensch-Maschine Interaktion. Dies ist ein Schritt hin zur einer engeren Integration der Fähigkeiten von Fahrer und Fahrzeug. Ein komplementärer Schritt wäre eine Erweiterung der Transparenz mentaler Zustände des Fahrers, wodurch eine wechselseitige Anpassung von Mensch und Maschine möglich wäre. Der letzte Teil dieser Arbeit diskutiert, wie diese Transparenz und weitere Voraussetzungen von Mensch-Maschine Kooperation erfüllt werden könnten, indem etwa Korrelate mentaler Zustände, insbesondere über das Blickverhalten, überwacht werden. Des Weiteren ergeben sich mit Blick auf zusätzliche kooperative Fähigkeiten neue Fragen über die Definition von Identität, sowie über die praktischen Konsequenzen von Mensch-Maschine Systemen, in denen ko-adaptive Agenten sensomotorische Prozesse vermittels einander ausüben können

Bodies of Seeing: A video ethnography of academic x-ray image interpretation training and professional vision in undergraduate radiology and radiography education

This thesis reports on a UK-based video ethnography of academic x-ray image interpretation training across two undergraduate courses in radiology and radiography. By studying the teaching and learning practices of the classroom, I initially explore the professional vision of x-ray image interpretation and how its relation to normal radiographic anatomy founds the practice of being ‘critical’. This criticality accomplishes a faculty of perceptual norms that is coded and organised and also, therefore, of a specific radiological vision. Professionals’ commitment to the cognitivist rhetoric of ‘looking at’/‘pattern recognition’ builds this critical perception, a perception that deepens in organisation when professionals endorse a ‘systematic approach’ that mediates matter-of-fact thoroughness and offers a helpful critical commentary towards the image. In what follows, I explore how x-ray image interpretation is constituted in case presentations. During training, x-ray images are treated with suspicion and as misleading and are aligned with a commitment to discursive contexts of ‘missed abnormality’, ‘interpretive risk’, and ‘technical error’. The image is subsequently constructed as ambiguous and that what is shown cannot be taken at face value. This interconnects with reenacting ideals around ‘seeing clearly’ that are explained through the teaching practices and material world of the academic setting and how, if misinterpretation is established, the ambiguity of the image is reduced by embodied gestures and technoscientific knowledge. By making this correction, the ambiguous image is reenacted and the misinterpretation of image content is explained. To conclude, I highlight how the professional vision of academic x-ray image interpretation prepares students for the workplace, shapes the classificatory interpretation of ab(normal) anatomy, manages ambiguity through embodied expectations and bodily norms, and cultivates body-machine relations

Face adaptation: behavioural and electrophysiological studies on the perception of eye gaze and gender

Whereas the investigation of perceptual aftereffects has a very long tradition in studies on low-level vision, the report and analysis of face-related high-level aftereffects is a very recent line of research. Webster et al. (2004, Nature) first showed a visual aftereffect on the perception of face gender by demonstrating that adaptation to male faces biased the subsequent classification of androgynous faces towards female gender. Similar adaptation effects have also been observed for one of the most important social signals: human eye gaze. Jenkins et al. (2006, Psychological Science) found that adaptation to gaze in one direction virtually eliminated participants’ ability to perceive smaller gaze deviations in the same direction. The present thesis further examined these high-level face aftereffects by determining the temporal characteristics of gaze adaptation and by analysing the neural correlates of eye gaze and gender adaptation. A behavioural study on the temporal decay of gaze adaptation effects shed further light on their basic characteristics: not only was the aftereffect surprisingly long-lasting, but its exponential decay revealed remarkable similarity with the time course of low-level adaptation effects. Further, in a series of event-related potential (ERP) studies it was found that the N170 was only marginally affected by both eye gaze and gender adaptation, whereas pronounced effects of both kinds of adaptation emerged in the P3 component with smaller amplitudes in response to test stimuli similar to the adaptation condition. Finally, gaze adaptation was found to affect ERPs in an earlier time interval ~250-350 ms which appeared to be sensitive to the discrimination between direct vs. averted gaze even when this was only an illusionary percept induced by adaptation. Together, these studies extend previous knowledge of the temporal parameters and the neural correlates of high-level face adaptation

Image and Evidence: The Study of Attention through the Combined Lenses of Neuroscience and Art

: Levy, EK 2012, ‘An artistic exploration of inattention blindness’, in Frontiers Hum Neurosci, vol. 5, ISSN=1662-5161.Full version unavailable due to 3rd party copyright restrictions.This study proposed that new insights about attention, including its phenomenon and pathology, would be provided by combining perspectives of the neurobiological discourse about attention with analyses of artworks that exploit the constraints of the attentional system. To advance the central argument that art offers a training ground for the attentional system, a wide range of contemporary art was analysed in light of specific tasks invoked. The kinds of cognitive tasks these works initiate with respect to the attentional system have been particularly critical to this research. Attention was explored within the context of transdisciplinary art practices, varied circumstances of viewing, new neuroscientific findings, and new approaches towards learning. Research for this dissertation required practical investigations in a gallery setting, and this original work was contextualised and correlated with pertinent neuroscientific approaches. It was also concluded that art can enhance public awareness of attention disorders and assist the public in discriminating between medical and social factors through questioning how norms of behaviour are defined and measured. This territory was examined through the comparative analysis of several diagnostic tests for attention deficit hyperactivity disorder (ADHD), through the adaptation of a methodology from economics involving patent citation in order to show market incentives, and through examples of data visualisation. The construction of an installation and collaborative animation allowed participants to experience first-hand the constraints on the attentional system, provoking awareness of our own “normal” physiological limitations. The embodied knowledge of images, emotion, and social context that are deeply embedded in art practices appeared to be capable of supplementing neuroscience’s understanding of attention and its disorders

The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study

Carminati MN, Knoeferle P. The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study. Presented at the Architectures and Mechanisms of Language and Processing (AMLaP), Riva del Garda, Italy

Redefining visual literacy in an era of visual overload: The use of reflective visual journals to expand students’ visual thinking

In an era in which “all media are mixed media” (Mitchell, 2002), visual information is central in interpersonal and mass communication. Despite this daily consumption of visual information, “digital natives” (Prensky, 2001) are not prepared to critically engage with images (Brumberger, 2016). Scholars in the field of visual literacy identified a curricular bias towards written texts (Elkins, 2007), and the need for more training of visual literacy in higher education (Metros & Woolsey, 2006). However, the discussion of visual literacy in higher education is dominated by studies that measure teaching strategies (Bowen, 2017; Johnston et al. 2017) but rarely discuss the meaning of visual literacy from a student perspective. Visual reflection is a learning experience that involves reading, writing, thinking, and feeling with and through images. This study investigates undergraduate students’ experience with visual reflection in a visual studies class through a phenomenographic approach to 29 visual journals and a thematic analysis of 9 semi-structured interviews with students. The objective is discussing the potential contribution of visual reflection to students’ multimodal literacies. This study contends that the promotion of visual reflection needs to be systematically implemented in all fields engaged in knowledge production as visual reflection enhances academic learning, fosters multimodal literacies, and promotes the visualization of knowledge

Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image type

Deep Learning models like Convolutional Neural Networks (CNN) are powerful image classifiers, but what factors determine whether they attend to similar image areas as humans do? While previous studies have focused on technological factors, little is known about the role of factors that affect human attention. In the present study, we investigated how the tasks used to elicit human attention maps interact with image characteristics in modulating the similarity between humans and CNN. We varied the intentionality of human tasks, ranging from spontaneous gaze during categorization over intentional gaze-pointing up to manual area selection. Moreover, we varied the type of image to be categorized, using either singular, salient objects, indoor scenes consisting of object arrangements, or landscapes without distinct objects defining the category. The human attention maps generated in this way were compared to the CNN attention maps revealed by explainable artificial intelligence (Grad-CAM). The influence of human tasks strongly depended on image type: For objects, human manual selection produced maps that were most similar to CNN, while the specific eye movement task has little impact. For indoor scenes, spontaneous gaze produced the least similarity, while for landscapes, similarity was equally low across all human tasks. To better understand these results, we also compared the different human attention maps to each other. Our results highlight the importance of taking human factors into account when comparing the attention of humans and CNN