Concepts, Attention, And The Contents Of Conscious Visual Experience

Ph.D. Thesis. University of Hawaiʻi at Mānoa 2018

THE COUPLING OF PERCEPTION AND ACTION IN REPRESENTATION

This thesis examines how the objects that we visually perceive in the world are coupled to the actions that we make towards them. For example, a whole hand grasp might be coupled with an object like an apple, but not with an object like a pea. It has been claimed that the coupling of what we see and what we do is not simply associative, but is fundamental to the way the brain represents visual objects. More than association, it is thought that when an object is seen (even if there is no intention to interact with it), there is a partial and automatic activation of the networks in the brain that plan actions (such as reaches and grasps). The central aim of this thesis was to investigate how specific these partial action plans might be, and how specific the properties of objects that automatically activate them might be. In acknowledging that perception and action are dynamically intertwining processes (such that in catching a butterfly the eye and the hand cooperate with a fluid and seamless efficiency), it was supposed that these couplings of perception and action in the brain might be loosely constrained. That is, they should not be rigidly prescribed (such that a highly specific action is always and only coupled with a specific object property) but they should instead involve fairly general components of actions that can adapt to different situations. The experimental work examined the automatic coupling of simplistic left and right actions (e.g. key presses) to pictures of oriented objects. Typically a picture of an object was shown and the viewer responded as fast as possible to some object property that was not associated with action (such as its colour). Of interest was how the performance of these left or right responses related to the task irrelevant left or right orientation of the object. The coupling of a particular response to a particular orientation could be demonstrated by the response performance (speed and accuracy). The more tightly coupled a response was to a particular object orientation, the faster and more accurate it was. The results supported the idea of loosely constrained action plans. Thus it appeared that a range of different actions (even foot responses) could be coupled with an object's orientation. These actions were coupled by default to an object's X-Z orientation (e.g. orientation in the depth plane). In further reflecting a loosely constrained perception-action mechanism, these couplings were shown to change in different situations (e.g. when the object moved towards the viewer, or when a key press made the object move in a predictable way). It was concluded that the kinds of components of actions that are automatically activated when viewing an object are not very detailed or fixed, but are initially quite general and can change and become more specific when circumstances demand it

Zero-gravity movement studies

The use of computer graphics to simulate the movement of articulated animals and mechanisms has a number of uses ranging over many fields. Human motion simulation systems can be useful in education, medicine, anatomy, physiology, and dance. In biomechanics, computer displays help to understand and analyze performance. Simulations can be used to help understand the effect of external or internal forces. Similarly, zero-gravity simulation systems should provide a means of designing and exploring the capabilities of hypothetical zero-gravity situations before actually carrying out such actions. The advantage of using a simulation of the motion is that one can experiment with variations of a maneuver before attempting to teach it to an individual. The zero-gravity motion simulation problem can be divided into two broad areas: human movement and behavior in zero-gravity, and simulation of articulated mechanisms

From surfaces to objects : Recognizing objects using surface information and object models.

This thesis describes research on recognizing partially obscured objects using surface information like Marr's 2D sketch ([MAR82]) and surface-based geometrical object models. The goal of the recognition process is to produce a fully instantiated object hypotheses, with either image evidence for each feature or explanations for their absence, in terms of self or external occlusion. The central point of the thesis is that using surface information should be an important part of the image understanding process. This is because surfaces are the features that directly link perception to the objects perceived (for normal "camera-like" sensing) and because surfaces make explicit information needed to understand and cope with some visual problems (e.g. obscured features). Further, because surfaces are both the data and model primitive, detailed recognition can be made both simpler and more complete. Recognition input is a surface image, which represents surface orientation and absolute depth. Segmentation criteria are proposed for forming surface patches with constant curvature character, based on surface shape discontinuities which become labeled segmentation- boundaries. Partially obscured object surfaces are reconstructed using stronger surface based constraints. Surfaces are grouped to form surface clusters, which are 3D identity-independent solids that often correspond to model primitives. These are used here as a context within which to select models and find all object features. True three-dimensional properties of image boundaries, surfaces and surface clusters are directly estimated using the surface data. Models are invoked using a network formulation, where individual nodes represent potential identities for image structures. The links between nodes are defined by generic and structural relationships. They define indirect evidence relationships for an identity. Direct evidence for the identities comes from the data properties. A plausibility computation is defined according to the constraints inherent in the evidence types. When a node acquires sufficient plausibility, the model is invoked for the corresponding image structure.Objects are primarily represented using a surface-based geometrical model. Assemblies are formed from subassemblies and surface primitives, which are defined using surface shape and boundaries. Variable affixments between assemblies allow flexibly connected objects. The initial object reference frame is estimated from model-data surface relationships, using correspondences suggested by invocation. With the reference frame, back-facing, tangential, partially self-obscured, totally self-obscured and fully visible image features are deduced. From these, the oriented model is used for finding evidence for missing visible model features. IT no evidence is found, the program attempts to find evidence to justify the features obscured by an unrelated object. Structured objects are constructed using a hierarchical synthesis process. Fully completed hypotheses are verified using both existence and identity constraints based on surface evidence. Each of these processes is defined by its computational constraints and are demonstrated on two test images. These test scenes are interesting because they contain partially and fully obscured object features, a variety of surface and solid types and flexibly connected objects. All modeled objects were fully identified and analyzed to the level represented in their models and were also acceptably spatially located. Portions of this work have been reported elsewhere ([FIS83], [FIS85a], [FIS85b], [FIS86]) by the author

THE POTENTIATION OF ACTIONS BY VISUAL OBJECTS

This thesis examines the relation between visual objects and the actions they afford. It is proposed that viewing an object results in the potentiation of the actions that can be made towards it. The proposal is consistent with neurophysiological evidence that suggests that no clear divide exists between visual and motor representation in the dorsal visual pathway, a processing stream that neuropsychological evidence strongly implicates in the visual control of actions. The experimental work presented examines motor system involvement in visual representation when no intention to perform a particular action is present. It is argued that the representation of action-relevant visual object properties, such as size and orientation, has a motor component. Thus representing the location of a graspable object involves representations of the motor commands necessary to bring the hand to the object. The proposal was examined in a series of eight experiments that employed a Stimulus- Response Compatibility paradigm in which the relation between responses and stimulus properties was never made explicit. Subjects had to make choice reaction time responses that mimicked a component of an action that a viewed object afforded. The action-relevant stimulus property was always irrelevant to response determination and consisted of components of the reach and grasp movement. The results found are not consistent with explanations based on the abstract coding of stimulus-response properties and strongly implicate the involvement of the action system. They provide evidence that merely viewing an object results in the activation of the motor patterns necessary to interact with them. The actions an object affords are an intrinsic part of its visual representation, not merely on account of the association between objects and familiar actions but because the motor system is directly involved in the representation of visuo-spatial object properties

Visual recognition of objects : behavioral, computational, and neurobiological aspects

I surveyed work on visual object recognition and perception. In animals, vision has been studied mainly on the behavioral and neurobiological levels. Behavioral data typically show what the visual system, by itself or together with the rest of the organism, is capable of. They show, for example, that humans can recognie objects regardless of size and position, but that rotated objects pose problems. Important insights into the organization of behavior have also been provided by people who suffered localized brain damage. We have learned that the brain is divided into areas subserving different and relatively well-defined behaviors. The visual system itself is also organized in different subsystems; the visual cortex alone contains nearly twenty maps of the visual field. And individual neurons respond selectively to visual stimuli, e.g., the orientation of line segments, color, direction of motion, and, most intriguingly, faces. The question is how the actions of all these neurons produce the behavior we observe. How do neurons represent the shape of objects such that they can be recognized? Before we can answer the question, we have to understand the computational aspect of shape representation, the nature of the problem as it were. Many methods for representing shape have been explored, mainly by computer scientists, but so far no satisfactory answers have been found

A cybernetic development of epistemology and observation applied to objects in space and time (as seen in architecture)

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This Thesis proposes a new epistemological ontology which has two peculiar characteristics: Objects in its Universe are formulated as being self-observers (i. e. reflexive); and the nature of observation of Objects by others is shown to contain the logic for computing relationships between Objects in the Universe. This Universe is non-hierarchical, and permits of mutually contradictory beliefs about its Objects to be simultaneously held by different observers. The logic by which observers construct hierarchies in the Universe is shown to need only one variable in order to operate, and to operate from the oscillatory nature of the self-observing Objects producing. a sense of local time in both observer, and observed Objects; the times of which must temporarily come together for observations to be made. Using these notions of Objects and observations, a means, based on the potential for observers to construct 0 hierarchies, is found for analysing arguments, and (potentially) for the improvement of computer performance. A way is described for the representation of observations of Objects to be made, and a conversational idiom is established to account for communication between different observers. The views put forward in this Thesis are demonstrated by various experiments, stories, and references