Action‐oriented Perception

: When I throw a ball at you, do you see it as catch‐able? Do we perceive objects as edible, climbable or Q‐able in general? One could argue that it is just a manner of speaking to say so: we do not really see an object as edible, we only infer on the basis of its other properties that it is. I argue that whether or not an object is edible or climbable is indeed represented perceptually: we see objects as edible, and do not just believe that they are. My argument proceeds in two steps. First, I argue that in order to perform an action Q with respect to an object, we need to represent this object as Q‐able and, second, I argue that we represent objects as having these properties perceptually

What is action-oriented perception?

Contemporary scientific and philosophical literature on perception often focuses on the relationship between perception and action, emphasizing the ways in which perception can be understood as geared towards action or ‘action-oriented’. In this paper I provide a framework within which to classify approaches to action-oriented perception, and I highlight important differences between the distinct approaches. I show how talk of perception as action-oriented can be applied to the evolutionary history of perception, neural or psychological perceptual mechanisms, the semantic content or phenomenal character of perceptual states, or to the metaphysical nature of perception. I argue that there are no straightforward inferences from one kind of action-oriented perception to another. Using this framework and its insights, I then explore the notion of action-oriented perceptual representation which plays a key role in some approaches to embodied cognitive science. I argue that the concept of action-oriented representation proposed by Clark and Wheeler is less straightforward than it might seem, because it seems to require both that the mechanisms of perceptual representation are action-oriented and that the content of these perceptual representations are action-oriented. Given that neither of these claims can be derived from the other, proponents of action-oriented representation owe us separate justification for each claim. I will argue that such justifications are not forthcoming in the literature, and that attempts to reconstruct them run into trouble: the sorts of arguments offered for the representational mechanisms being action-oriented seem to undermine the sorts of arguments offered for the representational content being action-oriented, and vice-versa

Action-based effects on music perception

The classical, disembodied approach to music cognition conceptualizes action and perception as separate, peripheral processes. In contrast, embodied accounts of music cognition emphasize the central role of the close coupling of action and perception. It is a commonly established fact that perception spurs action tendencies. We present a theoretical framework that captures the ways in which the human motor system and its actions can reciprocally influence the perception of music. The cornerstone of this framework is the common coding theory, postulating a representational overlap in the brain between the planning, the execution, and the perception of movement. The integration of action and perception in so-called internal models is explained as a result of associative learning processes. Characteristic of internal models is that they allow intended or perceived sensory states to be transferred into corresponding motor commands (inverse modeling), and vice versa, to predict the sensory outcomes of planned actions (forward modeling). Embodied accounts typically refer to inverse modeling to explain action effects on music perception (Leman, 2007). We extend this account by pinpointing forward modeling as an alternative mechanism by which action can modulate perception. We provide an extensive overview of recent empirical evidence in support of this idea. Additionally, we demonstrate that motor dysfunctions can cause perceptual disabilities, supporting the main idea of the paper that the human motor system plays a functional role in auditory perception. The finding that music perception is shaped by the human motor system and its actions suggests that the musical mind is highly embodied. However, we advocate for a more radical approach to embodied (music) cognition in the sense that it needs to be considered as a dynamical process, in which aspects of action, perception, introspection, and social interaction are of crucial importance

An information-theoretic on-line update principle for perception-action coupling

Inspired by findings of sensorimotor coupling in humans and animals, there has recently been a growing interest in the interaction between action and perception in robotic systems [Bogh et al., 2016]. Here we consider perception and action as two serial information channels with limited information-processing capacity. We follow [Genewein et al., 2015] and formulate a constrained optimization problem that maximizes utility under limited information-processing capacity in the two channels. As a solution we obtain an optimal perceptual channel and an optimal action channel that are coupled such that perceptual information is optimized with respect to downstream processing in the action module. The main novelty of this study is that we propose an online optimization procedure to find bounded-optimal perception and action channels in parameterized serial perception-action systems. In particular, we implement the perceptual channel as a multi-layer neural network and the action channel as a multinomial distribution. We illustrate our method in a NAO robot simulator with a simplified cup lifting task.Comment: 8 pages, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS

Angular Scale Expansion Theory And The Misperception Of Egocentric Distance In Locomotor Space

Perception is crucial for the control of action, but perception need not be scaled accurately to produce accurate actions. This paper reviews evidence for an elegant new theory of locomotor space perception that is based on the dense coding of angular declination so that action control may be guided by richer feedback. The theory accounts for why so much direct-estimation data suggests that egocentric distance is underestimated despite the fact that action measures have been interpreted as indicating accurate perception. Actions are calibrated to the perceived scale of space and thus action measures are typically unable to distinguish systematic (e.g., linearly scaled) misperception from accurate perception. Whereas subjective reports of the scaling of linear extent are difficult to evaluate in absolute terms, study of the scaling of perceived angles (which exist in a known scale, delimited by vertical and horizontal) provides new evidence regarding the perceptual scaling of locomotor space

A perception-action strategy for hummingbirds

Many human and animal tasks are thought to be controlled with the tau informational variable. It is widely accepted that controlling the rate of change of tau during decelerative tasks, such as when braking or landing, is one common perceptual control strategy. However, many tasks require accelerating before decelerating to a goal, such as reaching. An advancement of tau theory shows how a single action formula may be used to control the full action unit from initiation to peak velocity, and to rest at the goal, with the same perceptual tau information as before and accounting for the same decelerative kinematics as before. Here, we test the theory against data from high-speed video of a hummingbird flying to its flower feeder. We find that the theory accounts for 97% of the variance in the data, and thus supports it

The reentry hypothesis: linking eye movements to visual perception

Cortical organization of vision appears to be divided into perception and action. Models of vision have generally assumed that eye movements serve to select a scene for perception, so action and perception are sequential processes. We suggest a less distinct separation. According to our model, occulomotor areas responsible for planning an eye movement, such as the frontal eye field, influence perception prior to the eye movement. The activity reflecting the planning of an eye movement reenters the ventral pathway and sensitizes all cells within the movement field so the planned action determines perception. We demonstrate the performance of the computational model in a visual search task that demands an eye movement toward a target

Controlled Interaction: Strategies For Using Virtual Reality To Study Perception

Immersive virtual reality systems employing head-mounted displays offer great promise for the investigation of perception and action, but there are well-documented limitations to most virtual reality systems. In the present article, we suggest strategies for studying perception/action interactions that try to depend on both scale-invariant metrics (such as power function exponents) and careful consideration of the requirements of the interactions under investigation. New data concerning the effect of pincushion distortion on the perception of surface orientation are presented, as well as data documenting the perception of dynamic distortions associated with head movements with uncorrected optics. A review of several successful uses of virtual reality to study the interaction of perception and action emphasizes scale-free analysis strategies that can achieve theoretical goals while minimizing assumptions about the accuracy of virtual simulations

PAMPC: Perception-Aware Model Predictive Control for Quadrotors

We present the first perception-aware model predictive control framework for quadrotors that unifies control and planning with respect to action and perception objectives. Our framework leverages numerical optimization to compute trajectories that satisfy the system dynamics and require control inputs within the limits of the platform. Simultaneously, it optimizes perception objectives for robust and reliable sens- ing by maximizing the visibility of a point of interest and minimizing its velocity in the image plane. Considering both perception and action objectives for motion planning and control is challenging due to the possible conflicts arising from their respective requirements. For example, for a quadrotor to track a reference trajectory, it needs to rotate to align its thrust with the direction of the desired acceleration. However, the perception objective might require to minimize such rotation to maximize the visibility of a point of interest. A model-based optimization framework, able to consider both perception and action objectives and couple them through the system dynamics, is therefore necessary. Our perception-aware model predictive control framework works in a receding-horizon fashion by iteratively solving a non-linear optimization problem. It is capable of running in real-time, fully onboard our lightweight, small-scale quadrotor using a low-power ARM computer, to- gether with a visual-inertial odometry pipeline. We validate our approach in experiments demonstrating (I) the contradiction between perception and action objectives, and (II) improved behavior in extremely challenging lighting conditions