Keeping track of objects while exploring an informationally impoverished environment: Local deictic versus global spatial strategies

This study investigates a new experimental paradigm called the Modified Traveling Salesman Problem (Bullot & Droulez, submitted). This task requires subjects to visit once and only once n invisible targets in a 2D display, using a virtual vehicle controlled by the subject. Subjects can only see the directions of the targets from the current location of the vehicle, displayed by a set of oriented segments that can be viewed inside a circular window surrounding the vehicle. Two conditions were compared. In the “allocentric” condition, subjects see the vehicle move across the screen and change orientation under their command. The “egocentric” condition is similar except for how the information is provided: the position and orientation of the vehicle icon remains fixed at the center of the screen and only target directions, as indicated by the oriented segments, change as the subject “moves” the vehicle. The unexpected finding was that this task can be performed, in either condition, for up to 10 targets. We consider two possible strategies that might be used, a location-based strategy and a segment strategy. The location-based strategy relies on spatial memory and attempts to infer the locations of all the targets. The segment strategy is more local and focuses on the directional segments themselves, keeping track of the ones that represent already-visited targets. A number of observations suggest that the segment strategy was used, at least for larger numbers of targets. According to our hypothesis, keeping track of the segments requires one to use indexical reference for associating the segments with their status in the task - given by current status predicates Visited(x) or Not-visited(x) -, perhaps using visual indexes (Pylyshyn, 2001), deictic pointers (Ballard et al., 1997), or object files (Kahneman et al, 1992)

Model-based Cognitive Neuroscience: Multifield Mechanistic Integration in Practice

Autonomist accounts of cognitive science suggest that cognitive model building and theory construction (can or should) proceed independently of findings in neuroscience. Common functionalist justifications of autonomy rely on there being relatively few constraints between neural structure and cognitive function (e.g., Weiskopf, 2011). In contrast, an integrative mechanistic perspective stresses the mutual constraining of structure and function (e.g., Piccinini & Craver, 2011; Povich, 2015). In this paper, I show how model-based cognitive neuroscience (MBCN) epitomizes the integrative mechanistic perspective and concentrates the most revolutionary elements of the cognitive neuroscience revolution (Boone & Piccinini, 2016). I also show how the prominent subset account of functional realization supports the integrative mechanistic perspective I take on MBCN and use it to clarify the intralevel and interlevel components of integration

Do objects in working memory compete with objects in perception?

It is generally assumed that "perceptual object" is the basic unit for processing visual information and that only a small number of objects can be either perceptually selected or encoded in working memory (WM) at one time. This raises the question whether the same resource is used when objects are selected and tracked as when they are held in WM. In two experiments, we measured dual-task interference between a memory task and a Multiple Object Tracking task. The WM tasks involve explicit, implicit, or no spatial processing. Our results suggest there is no resource competition between working memory and perceptual selection except when the WM task requires encoding spatial properties

Keeping track of objects while exploring a spatial layout with partial cues: Location-based and direction-based strategies [Abstract]

Last year at VSS, Bullot, Droulez & Pylyshyn (2003) reported studies using a Modified Traveling Salesman Paradigm (MTSP) in which a virtual vehicle had to visit up to 10 targets once and only once, and in which the invisible targets were identified only by line segments pointing from the vehicle toward each target. We hypothesized that subjects used two distinct strategies: A “location-based strategy”, which kept track of where targets were located in screen coordinates, and a “segment-based strategy” that kept track of which segments corresponded to visited targets. We report new studies that further explore these two strategies. Subjects passively observed a computer-controlled virtual vehicle that visited a number of targets. Two forms of display were used: an “Allocentric” display, in which the vehicle moved and the targets remained fixed in screen coordinates, and an “Egocentric” display, in which the vehicle's position on the screen remained fixed while the targets moved -- as if the environment were being viewed by an observer on the vehicle. At the end of each trial, the directional segments were extended to the edge of the screen and subjects were asked to perform two tasks by referring to these segments. In the “status task” observers had to indicate for each segment whether the corresponding target had been visited or not. In the “locating task” they had to locate each target along its directional segment. Performance on these two tasks measures the use of the two hypothesized strategies (segment-based and location-based). Results showed that observers do well on the status task with 4 or 6 targets in both display conditions, but do poorly on the locating task, especially in the egocentric condition when there are more targets. These results are consistent with the hypothesis that in the egocentric condition the MTSP task is carried out primarily by segment-tracking, which can be viewed as a deictic strategy (Ballard et al., 1997; Pylyshyn, 2001)

Verifying Different-modality Properties for Concepts Produces Switching Costs

According to perceptual symbol systems (Barsalou, 1999), sensory-motor simulations underlie the representation of concepts. It follows that sensory-motor phenomena should arise in conceptual processing. Previous studies have shown that switching from one modality to another during perceptual processing incurs a processing cost. If perceptual simulation underlies conceptual processing, then verifying the properties of concepts should exhibit a switching cost as well. For example, verifying a property in the auditory modality (e.g., BLENDER-loud) should be slower after verifying a property in a different modality (e.g., CRANBERRIES-tart) than in the same modality (e.g., LEAVES-rustling). Only words were presented to subjects, and there were no instructions to use imagery. Nevertheless switching modalities incurred a cost, analogous to switching modalities in perception. A second experiment showed that this effect was not due to associative priming between properties in the same modality. These results support the hypothesis that perceptual simulation underlies conceptual processing

Temporal estimation with two moving objects: overt and covert pursuit

The current study examined temporal estimation in a prediction motion task where participants were cued to overtly pursue one of two moving objects, which could either arrive first, i.e., shortest [time to contact (TTC)] or second (i.e., longest TTC) after a period of occlusion. Participants were instructed to estimate TTC of the first-arriving object only, thus making it necessary to overtly pursue the cued object while at the same time covertly pursuing the other (non-cued) object. A control (baseline) condition was also included in which participants had to estimate TTC of a single, overtly pursued object. Results showed that participants were able to estimate the arrival order of the two objects with very high accuracy irrespective of whether they had overtly or covertly pursued the first-arriving object. However, compared to the single-object baseline, participants’ temporal estimation of the covert object was impaired when it arrived 500 ms before the overtly pursued object. In terms of eye movements, participants exhibited significantly more switches in gaze location during occlusion from the cued to the non-cued object but only when the latter arrived first. Still, comparison of trials with and without a switch in gaze location when the non-cued object arrived first indicated no advantage for temporal estimation. Taken together, our results indicate that overt pursuit is sufficient but not necessary for accurate temporal estimation. Covert pursuit can enable representation of a moving object’s trajectory and thereby accurate temporal estimation providing the object moves close to the overt attentional focus

Telephone conversation impairs sustained visual attention via a central bottleneck

Recent research has shown that holding telephone conversations disrupts one's driving ability. We asked whether this effect could be attributed to a visual attention impairment. In Experiment 1, participants conversed on a telephone or listened to a narrative while engaged in multiple object tracking (MOT), a task requiring sustained visual attention. We found that MOT was disrupted in the telephone conversation condition, relative to single-task MOT performance, but that listening to a narrative had no effect. In Experiment 2, we asked which component of conversation might be interfering with MOT performance. We replicated the conversation and single-task conditions of Experiment 1 and added two conditions in which participants heard a sequence of words over a telephone. In the shadowing condition, participants simply repeated each word in the sequence. In the generation condition, participants were asked to generate a new word based on each word in the sequence. Word generation interfered with MOT performance, but shadowing did not. The data indicate that telephone conversation disrupts attention at a central stage, the act of generating verbal stimuli, rather than at a peripheral stage, such as listening or speaking