Episodic visual cognition: Implications for object and short-term recognition

Poth CH. Episodic visual cognition: Implications for object and short-term recognition. Bielefeld: Universität Bielefeld; 2017.A great part of human behavior is based on visual cognition, the processing of visual information about external objects. For goal-directed behavior, two functions of visual cognition seem especially important. The first one is object recognition. Objects in the environment must be identified as belonging to an object category, so that they can be used to accomplish a given task. The second function is short-term recognition. It must be recognized whether an object in the environment has been viewed recently, so that current behavior involving the object can be related to previous behavior. Both functions share a common constraint: They must be fulfilled across distinct episodes of visual processing, which are interrupted by changes in processing demands. For object recognition, visual processing episodes lead to a problem of selective integration. That is, it must be decided whether object information from the current episode should update and thus be integrated with object representations from the previous episode. Alternatively, object representations from two successive episodes are retained separately. This decision is critical. Updating and integration should enable a cumulative and fast object recognition. However, integration should also conceal object changes across episodes by leaving no separate representations that can be compared. Separation should improve change perception but impair object recognition, because limited visual processing resources for object recognition must be split between the separate representations. For short-term recognition visual processing episodes lead to a problem of matching. That is, an object from the current episode must be matched against object representations, not only from the previous but from several episodes in the recent past (irrespective of whether objects are categorized). The overarching goal of the present dissertation is to make a first step in understanding how the mechanisms underlying object recognition and short-term recognition operate across visual processing episodes, and how they solve the two problems. In five empirical studies, we investigated key issues that must be addressed before a theoretical account of object and short-term recognition across visual processing episodes can be given. The first three studies focused on object recognition across visual processing episodes of eye fixations. Fixations are periods of visual information uptake, in which the eyes stand relatively still. They are separated by rapid saccadic eye movements. Saccades are necessary for object recognition, because they direct the central fovea of the eye's retina at interesting objects, allowing high-acuity inspection. However, saccades also disrupt visual input and displace and alter the retinal images of objects. Therefore, saccades dissect visual information processing into distinct episodes of fixations, which the mechanisms for object recognition must accommodate. In two studies (Poth, Herwig, & Schneider, 2015; Poth & Schneider, 2016a), we investigated how the selective integration problem is solved to support object recognition across successive fixations. We assessed the recent hypothesis (Schneider, 2013) that the problem is solved by a mechanism testing for correspondence (``object continuity'') between an object before and after a saccade. If object correspondence is established, the object before and after the saccade should be integrated into a common representation. In contrast, if object correspondence is broken, the object before and after the saccade should be represented separately. Separation should allow to compare the two representations, improving the discrimination of transsaccadic object displacements. At the same time, however, object recognition of the object after the saccade should be impaired, because the necessary visual processing resources had to be split between the two representations. Results were consistent with this hypothesis. Breaking object correspondence by briefly blanking an object after a saccade to it improved the discrimination of displacements of the object but impaired object recognition. Thus, the object correspondence mechanism seems to impact on object recognition after the saccade. Further experiments investigated the nature of object correspondence. They showed that object recognition was impaired when object correspondence was broken by changing an object's contrast-polarity (and luminance), its color-and-luminance, and its color alone. Together with the initial finding, this indicates that object correspondence is based on spatiotemporal as well as on the surface features of objects. In the third study (Poth & Schneider, 2016b, submitted), we went on to test the limits of object recognition across saccades. Because object recognition relies on limited visual processing resources, it can only be achieved for a few objects at a time. Here, we examined if different objects must compete for these resources across saccades. If this was the case, visual processing after a saccade would be slowed down as more and more objects are viewed before the saccade. Our findings show that this is the case, but only if the objects are task-relevant. Therefore, the findings support a key prediction of a recent theory, namely that the importance of an object representation determines whether it will survive a saccade and take up limited processing resources afterwards (Schneider, 2013). With the fourth study (Poth & Schneider, 2016c), we turned from the processing episodes of successive eye fixations to those defined by appearing and disappearing objects and associated task-requirements. We asked about the relationship between the mechanisms underlying object recognition and those underlying short-term recognition. Visual processing for object recognition is assumed to be complete when an object has entered a limited-capacity visual working memory, where the object becomes available for being reported. We investigated if encoding into visual working memory is not only required for object recognition in the current episode, but also for short-term recognition in upcoming episodes. Supporting this notion, we found that objects that supposedly had not reached visual working memory were not available for later short-term recognition. This finding argues that the initial steps of visual processing before encoding into visual working memory are not sufficient for short-term recognition in later episodes. Therefore, visual working memory may contribute to the solution of the matching problem by limiting the amount of information considered in a short-term recognition task. Finally, in the fifth study (Poth & Schneider, 2016d, submitted), we investigated short-term recognition further, asking how short-term recognition in a later processing episode can be prepared in advance. We assessed how prioritizing among objects represented in visual working memory impacts on two distinct components of performance in an upcoming short-term recognition task. Our results showed that such a prioritization improves memory-retention in visual working memory but also accelerates visual processing of objects for short-term recognition in a future episode. This indicates that changes in processing priorities contribute to ongoing solutions of the matching problem of short-term recognition. Taken together, the five studies show how mechanisms of object and short-term recognition address specific problems arising from the dissection of visual processing into distinct episodes. As such, the studies implicate visual processing episodes as a source of problems for object and short-term recognition, which is neglected in most contemporary research. Conversely, however, the studies also invite speculation about the functional value of visual processing episodes for visual cognition

Episodic Short-Term Recognition Requires Encoding into Visual Working Memory: Evidence from Probe Recognition after Letter Report

Poth CH, Schneider WX. Episodic Short-Term Recognition Requires Encoding into Visual Working Memory: Evidence from Probe Recognition after Letter Report. Frontiers in Psychology. 2016;7: 1440.Human vision is organized in discrete processing episodes (e.g., eye fixations or task-steps). Object information must be transmitted across episodes to enable episodic short-term recognition: recognizing whether a current object has been seen in a previous episode. We ask whether episodic short-term recognition presupposes that objects have been encoded into capacity-limited visual working memory (VWM), which retains visual information for report. Alternatively, it could rely on the activation of visual features or categories that occurs before encoding into VWM. We assessed the dependence of episodic short-term recognition on VWM by a new paradigm combining letter report and probe recognition. Participants viewed displays of 10 letters and reported as many as possible after a retention interval (whole report). Next, participants viewed a probe letter and indicated whether it had been one of the 10 letters (probe recognition). In Experiment 1, probe recognition was more accurate for letters that had been encoded into VWM (reported letters) compared with non-encoded letters (non-reported letters). Interestingly, those letters that participants reported in their whole report had been near to one another within the letter displays. This suggests that the encoding into VWM proceeded in a spatially clustered manner. In Experiment 2, participants reported only one of 10 letters (partial report) and probes either referred to this letter, to letters that had been near to it, or far from it. Probe recognition was more accurate for near than for far letters, although none of these letters had to be reported. These findings indicate that episodic short-term recognition is constrained to a small number of simultaneously presented objects that have been encoded into VWM

Breaking object correspondence across saccadic eye movements deteriorates object recognition

Poth CH, Herwig A, Schneider WX. Breaking object correspondence across saccadic eye movements deteriorates object recognition. Frontiers in Systems Neuroscience. 2015;9: 176.Visual perception is based on information processing during periods of eye fixations that are interrupted by fast saccadic eye movements. The ability to sample and relate information on task-relevant objects across fixations implies that correspondence between presaccadic and postsaccadic objects is established. Postsaccadic object information usually updates and overwrites information on the corresponding presaccadic object. The presaccadic object representation is then lost. In contrast, the presaccadic object is conserved when object correspondence is broken. This helps transsaccadic memory but it may impose attentional costs on object recognition. Therefore, we investigated how breaking object correspondence across the saccade affects postsaccadic object recognition. In Experiment 1, object correspondence was broken by a brief postsaccadic blank screen. Observers made a saccade to a peripheral object which was displaced during the saccade. This object reappeared either immediately after the saccade or after the blank screen. Within the postsaccadic object, a letter was briefly presented (terminated by a mask). Observers reported displacement direction and letter identity in different blocks. Breaking object correspondence by blanking improved displacement identification but deteriorated postsaccadic letter recognition. In Experiment 2, object correspondence was broken by changing the object’s contrast-polarity. There were no object displacements and observers only reported letter identity. Again, breaking object correspondence deteriorated postsaccadic letter recognition. These findings identify transsaccadic object correspondence as a key determinant of object recognition across the saccade. This is in line with the recent hypothesis that breaking object correspondence results in separate representations of presaccadic and postsaccadic objects which then compete for limited attentional processing resources (Schneider, 2013). Postsaccadic object recognition is then deteriorated because less resources are available for processing postsaccadic objects