Probabilistic modeling of eye movement data during conjunction search via feature-based attention

Where the eyes fixate during search is not random; rather, gaze reflects the combination of information about the target and the visual input. It is not clear, however, what information about a target is used to bias the underlying neuronal responses. We here engage subjects in a variety of simple conjunction search tasks while tracking their eye movements. We derive a generative model that reproduces these eye movements and calculate the conditional probabilities that observers fixate, given the target, on or near an item in the display sharing a specific feature with the target. We use these probabilities to infer which features were biased by top-down attention: Color seems to be the dominant stimulus dimension for guiding search, followed by object size, and lastly orientation. We use the number of fixations it took to find the target as a measure of task difficulty. We find that only a model that biases multiple feature dimensions in a hierarchical manner can account for the data. Contrary to common assumptions, memory plays almost no role in search performance. Our model can be fit to average data of multiple subjects or to individual subjects. Small variations of a few key parameters account well for the intersubject differences. The model is compatible with neurophysiological findings of V4 and frontal eye fields (FEF) neurons and predicts the gain modulation of these cells

Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli

In natural vision both stimulus features and task-demands affect an observer's attention. However, the relationship between sensory-driven (“bottom-up”) and task-dependent (“top-down”) factors remains controversial: Can task-demands counteract strong sensory signals fully, quickly, and irrespective of bottom-up features? To measure attention under naturalistic conditions, we recorded eye-movements in human observers, while they viewed photographs of outdoor scenes. In the first experiment, smooth modulations of contrast biased the stimuli's sensory-driven saliency towards one side. In free-viewing, observers' eye-positions were immediately biased toward the high-contrast, i.e., high-saliency, side. However, this sensory-driven bias disappeared entirely when observers searched for a bull's-eye target embedded with equal probability to either side of the stimulus. When the target always occurred in the low-contrast side, observers' eye-positions were immediately biased towards this low-saliency side, i.e., the sensory-driven bias reversed. Hence, task-demands do not only override sensory-driven saliency but also actively countermand it. In a second experiment, a 5-Hz flicker replaced the contrast gradient. Whereas the bias was less persistent in free viewing, the overriding and reversal took longer to deploy. Hence, insufficient sensory-driven saliency cannot account for the bias reversal. In a third experiment, subjects searched for a spot of locally increased contrast (“oddity”) instead of the bull's-eye (“template”). In contrast to the other conditions, a slight sensory-driven free-viewing bias prevails in this condition. In a fourth experiment, we demonstrate that at known locations template targets are detected faster than oddity targets, suggesting that the former induce a stronger top-down drive when used as search targets. Taken together, task-demands can override sensory-driven saliency in complex visual stimuli almost immediately, and the extent of overriding depends on the search target and the overridden feature, but not on the latter's free-viewing saliency

Time course of target recognition in visual search

Visual search is a ubiquitous task of great importance: it allows us to quickly find the objects that we are looking for. During active search for an object (target), eye movements are made to different parts of the scene. Fixation locations are chosen based on a combination of information about the target and the visual input. At the end of a successful search, the eyes typically fixate on the target. But does this imply that target identification occurs while looking at it? The duration of a typical fixation (~170ms) and neuronal latencies of both the oculomotor system and the visual stream indicate that there might not be enough time to do so. Previous studies have suggested the following solution to this dilemma: the target is identified extrafoveally and this event will trigger a saccade towards the target location. However this has not been experimentally verified. Here we test the hypothesis that subjects recognize the target before they look at it using a search display of oriented colored bars. Using a gaze-contingent real-time technique, we prematurely stopped search shortly after subjects fixated the target. Afterwards, we asked subjects to identify the target location. We find that subjects can identify the target location even when fixating on the target for less than 10ms. Longer fixations on the target do not increase detection performance but increase confidence. In contrast, subjects cannot perform this task if they are not allowed to move their eyes. Thus, information about the target during conjunction search for colored oriented bars can, in some circumstances, be acquired at least one fixation ahead of reaching the target. The final fixation serves to increase confidence rather then performance, illustrating a distinct role of the final fixation for the subjective judgment of confidence rather than accuracy

The relation of phase noise and luminance contrast to overt attention in complex visual stimuli

Models of attention are typically based on difference maps in low-level features but neglect higher order stimulus structure. To what extent does higher order statistics affect human attention in natural stimuli? We recorded eye movements while observers viewed unmodified and modified images of natural scenes. Modifications included contrast modulations (resulting in changes to first- and second-order statistics), as well as the addition of noise to the Fourier phase (resulting in changes to higher order statistics). We have the following findings: (1) Subjects' interpretation of a stimulus as a “natural” depiction of an outdoor scene depends on higher order statistics in a highly nonlinear, categorical fashion. (2) Confirming previous findings, contrast is elevated at fixated locations for a variety of stimulus categories. In addition, we find that the size of this elevation depends on higher order statistics and reduces with increasing phase noise. (3) Global modulations of contrast bias eye position toward high contrasts, consistent with a linear effect of contrast on fixation probability. This bias is independent of phase noise. (4) Small patches of locally decreased contrast repel eye position less than large patches of the same aggregate area, irrespective of phase noise. Our findings provide evidence that deviations from surrounding statistics, rather than contrast per se, underlie the well-established relation of contrast to fixation

Is bottom-up attention useful for object recognition?

A key problem in learning multiple objects from unlabeled images is that it is a priori impossible to tell which part of the image corresponds to each individual object, and which part is irrelevant clutter which is not associated to the objects. We investigate empirically to what extent pure bottom-up attention can extract useful information about the location, size and shape of objects from images and demonstrate how this information can be utilized to enable unsupervised learning of objects from unlabeled images. Our experiments demonstrate that the proposed approach to using bottom-up attention is indeed useful for a variety of applications

Controlling Volatility and Nonvolatility of Memristive Devices by Sn Alloying

Memristive devices have attracted significant attention due to their downscaling potential, low power operation, and fast switching performance. Their inherent properties make them suitable for emerging applications such as neuromorphic computing, in-memory computing, and reservoir computing. However, the different applications demand either volatile or nonvolatile operation. In this study, we demonstrate how compliance current and specific material choices can be used to control the volatility and nonvolatility of memristive devices. Especially, by mixing different materials in the active electrode, we gain additional design parameters that allow us to tune the devices for different applications. We found that alloying Ag with Sn stabilizes the nonvolatile retention regime in a reproducible manner. Additionally, our alloying approach improves the reliability, endurance, and uniformity of the devices. We attribute these advances to stabilization of the filament inside the switching medium by the inclusion of Sn in the filament structure. These advantageous properties of alloying were found by investigating a choice of six electrode materials (Ag, Cu, AgCu-1, AgCu-2, AgSn-1, AgSn-2) and three switching layers (SiO$_2$, Al$_2$O$_3$, HfO$_2$)

Multi-modal characterization and simulation of human epileptic circuitry

Temporal lobe epilepsy is the fourth most common neurological disorder with about 40% of patients not responding to pharmacological treatment. Increased cellular loss in the hippocampus is linked to disease severity and pathological phenotypes such as heightened seizure propensity. While the hippocampus is the target of therapeutic interventions such as temporal lobe resection, the impact of the disease at the cellular level remains unclear in humans. Here we show that properties of hippocampal granule cells change with disease progression as measured in living, resected hippocampal tissue excised from epilepsy patients. We show that granule cells increase excitability and shorten response latency while also enlarging in cellular volume, surface area and spine density. Single-cell RNA sequencing combined with simulations ascribe the observed electrophysiological changes to gradual modification in three key ion channel conductances: BK, Cav2.2 and Kir2.1. In a bio-realistic computational network model, we show that the changes related to disease progression bring the circuit into a more excitable state. In turn, we observe that by reversing these changes in the three key conductances produces a less excitable, early disease-like state. These results provide mechanistic understanding of epilepsy in humans and will inform future therapies such as viral gene delivery to reverse the course of the disorder

Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering

This publication is the Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering from July 6-8, 2022. The EG-ICE International Workshop on Intelligent Computing in Engineering brings together international experts working on the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolution of challenges such as supporting multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways. &nbsp

Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering

This publication is the Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering from July 6-8, 2022. The EG-ICE International Workshop on Intelligent Computing in Engineering brings together international experts working on the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolution of challenges such as supporting multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways. &nbsp