How sketches work: a cognitive theory for improved system design

Evidence is presented that in the early stages of design or composition the mental processes used by artists for visual invention require a different type of support from those used for visualising a nearly complete object. Most research into machine visualisation has as its goal the production of realistic images which simulate the light pattern presented to the retina by real objects. In contrast sketch attributes preserve the results of cognitive processing which can be used interactively to amplify visual thought. The traditional attributes of sketches include many types of indeterminacy which may reflect the artist's need to be "vague". Drawing on contemporary theories of visual cognition and neuroscience this study discusses in detail the evidence for the following functions which are better served by rough sketches than by the very realistic imagery favoured in machine visualising systems. 1. Sketches are intermediate representational types which facilitate the mental translation between descriptive and depictive modes of representing visual thought. 2. Sketch attributes exploit automatic processes of perceptual retrieval and object recognition to improve the availability of tacit knowledge for visual invention. 3. Sketches are percept-image hybrids. The incomplete physical attributes of sketches elicit and stabilise a stream of super-imposed mental images which amplify inventive thought. 4. By segregating and isolating meaningful components of visual experience, sketches may assist the user to attend selectively to a limited part of a visual task, freeing otherwise over-loaded cognitive resources for visual thought. 5. Sequences of sketches and sketching acts support the short term episodic memory for cognitive actions. This assists creativity, providing voluntary control over highly practised mental processes which can otherwise become stereotyped. An attempt is made to unite the five hypothetical functions. Drawing on the Baddeley and Hitch model of working memory, it is speculated that the five functions may be related to a limited capacity monitoring mechanism which makes tacit visual knowledge explicitly available for conscious control and manipulation. It is suggested that the resources available to the human brain for imagining nonexistent objects are a cultural adaptation of visual mechanisms which evolved in early hominids for responding to confusing or incomplete stimuli from immediately present objects and events. Sketches are cultural inventions which artificially mimic aspects of such stimuli in order to capture these shared resources for the different purpose of imagining objects which do not yet exist. Finally the implications of the theory for the design of improved machine systems is discussed. The untidy attributes of traditional sketches are revealed to include cultural inventions which serve subtle cognitive functions. However traditional media have many short-comings which it should be possible to correct with new technology. Existing machine systems for sketching tend to imitate nonselectively the media bound properties of sketches without regard to the functions they serve. This may prove to be a mistake. It is concluded that new system designs are needed in which meaningfully structured data and specialised imagery amplify without interference or replacement the impressive but limited creative resources of the visual brain

The Influence of the Dorsal Pathway on Enhanced Visual Processing

Overall our visual experience is such a seamless one that unless specifically told, we might never know that what we see is actually the visual system taking the very simple input provided by cells in the retina and constructing an image based on rules and calculations and algorithms neuroscientists have yet to fully uncover. This is an incredible feat given the plethora of visual stimuli within our environment, that this information is used to inform and plan actions, and if that wasnt enough, the visual system also has the capacity to selectively enhance certain aspects of visual processing if needs be. The research contained within this dissertation seeks to investigate how the dorsal visual pathway enhances both decision-making processes and visual stimuli presented near the hand. Our findings suggest that the formation of object representations in the dorsal pathway can include both ventral (colour, contrast) and dorsal (speed) stream features (chapters two and three), which in turn greatly speed decision-making processes within the dorsal pathway. In addition, contrast and speed are integrated automatically but purely ventral stream features, such as colour, require top-down attention to facilitate enhanced processing speeds (chapter three). In chapter four we find that visual processing near the hand is enhanced in a novel way. When the hand is nearby, orientation tuning is sharpened in a manner not consistent with either oculomotor-driven spatial or feature based attention. In addition, response variability is reduced when the hand is nearby, raising the possibility that enhanced processing near the hand maybe be driven by feedback from frontoparietal reaching and grasping regions. The research within this dissertation includes important new information regarding how the dorsal pathway can speed visual processing, and provides insight as to the next stage in understanding how we use vision for action

Top-down effects on early visual processing in humans: a predictive coding framework

An increasing number of human electroencephalography (EEG) studies examining the earliest component of the visual evoked potential, the so-called C1, have cast doubts on the previously prevalent notion that this component is impermeable to top-down effects. This article reviews the original studies that (i) described the C1, (ii) linked it to primary visual cortex (V1) activity, and (iii) suggested that its electrophysiological characteristics are exclusively determined by low-level stimulus attributes, particularly the spatial position of the stimulus within the visual field. We then describe conflicting evidence from animal studies and human neuroimaging experiments and provide an overview of recent EEG and magnetoencephalography (MEG) work showing that initial V1 activity in humans may be strongly modulated by higher-level cognitive factors. Finally, we formulate a theoretical framework for understanding top-down effects on early visual processing in terms of predictive coding

Modulation of Brain Activity by the Integration of Color into Dorsal Stream Object Files

Two superimposed surfaces of dots are perceived as separate objects when rotating in two different directions. When one surface is cued, there is a larger suppression of the attentional ERP components of the unattended surface than the attended surface when two objects are perceived versus when one object is perceived. We hypothesized that the strength of object-based attention was dependent on the differentiation of the two object representations. We tested this hypothesis by determining if two oppositely rotating superimposed surfaces of differing colors would produce a greater cueing effect than if the two surfaces were the same color. This additional color feature would allow for object files with stronger neural representation, leading to a greater suppression of the uncued surface in the task. It was found that there was a greater cueing effect in the bicolored condition compared to the unicolored condition both behaviorally and in event related potentials

Speaker Normalization Using Cortical Strip Maps: A Neural Model for Steady State Vowel Identification

Auditory signals of speech are speaker-dependent, but representations of language meaning are speaker-independent. Such a transformation enables speech to be understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitchindependent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model fits into an emerging model of auditory streaming and speech categorization. The auditory streaming and speaker normalization parts of the model both use multiple strip representations and asymmetric competitive circuits, thereby suggesting that these two circuits arose from similar neural designs. The normalized speech items are rapidly categorized and stably remembered by Adaptive Resonance Theory circuits. Simulations use synthesized steady-state vowels from the Peterson and Barney [J. Acoust. Soc. Am. 24, 175-184 (1952)] vowel database and achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models.National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Structural and functional differences in medial prefrontal cortex underlie distractibility and suppression deficits in ageing.

Older adults experience deficits in working memory (WM) that are acutely exacerbated by the presence of distracting information. Human neurophysiological studies have revealed that these changes are accompanied by a diminished ability to suppress visual cortical activity associated with task-irrelevant information. Although this is often attributed to deficits in top-down control from a prefrontal cortical source, this has not yet been directly demonstrated. Here we evaluate the neural basis of distraction's negative impact on WM and the impairment in neural suppression in older adults by performing structural and functional MRIs while older participants engage in tasks that require remembering relevant visual stimuli in the context of overlapping irrelevant stimuli. Analysis supports both an age-related distraction effect and neural suppression deficit, and extends our understanding by revealing an alteration in functional connectivity between visual cortices and a region in the default network, the medial prefrontal cortex (mPFC). Moreover, within the older population, the magnitude of WM distractibility and neural suppression are both associated with individual differences in cortical volume and activity of the mPFC, as well as its associated white-matter tracts