Risk-Sensitivity in Sensorimotor Control

Recent advances in theoretical neuroscience suggest that motor control can be considered as a continuous decision-making process in which uncertainty plays a key role. Decision-makers can be risk-sensitive with respect to this uncertainty in that they may not only consider the average payoff of an outcome, but also consider the variability of the payoffs. Although such risk-sensitivity is a well-established phenomenon in psychology and economics, it has been much less studied in motor control. In fact, leading theories of motor control, such as optimal feedback control, assume that motor behaviors can be explained as the optimization of a given expected payoff or cost. Here we review evidence that humans exhibit risk-sensitivity in their motor behaviors, thereby demonstrating sensitivity to the variability of “motor costs.” Furthermore, we discuss how risk-sensitivity can be incorporated into optimal feedback control models of motor control. We conclude that risk-sensitivity is an important concept in understanding individual motor behavior under uncertainty

Risk-Sensitive Optimal Feedback Control Accounts for Sensorimotor Behavior under Uncertainty

Many aspects of human motor behavior can be understood using optimality principles such as optimal feedback control. However, these proposed optimal control models are risk-neutral; that is, they are indifferent to the variability of the movement cost. Here, we propose the use of a risk-sensitive optimal controller that incorporates movement cost variance either as an added cost (risk-averse controller) or as an added value (risk-seeking controller) to model human motor behavior in the face of uncertainty. We use a sensorimotor task to test the hypothesis that subjects are risk-sensitive. Subjects controlled a virtual ball undergoing Brownian motion towards a target. Subjects were required to minimize an explicit cost, in points, that was a combination of the final positional error of the ball and the integrated control cost. By testing subjects on different levels of Brownian motion noise and relative weighting of the position and control cost, we could distinguish between risk-sensitive and risk-neutral control. We show that subjects change their movement strategy pessimistically in the face of increased uncertainty in accord with the predictions of a risk-averse optimal controller. Our results suggest that risk-sensitivity is a fundamental attribute that needs to be incorporated into optimal feedback control models

Risk-sensitivity.

<p>A. Results of the multilinear regression analysis of the low control cost conditions for subject number 5. The line shows the average motor command that the subject produces for a given position (blue - low noise level, yellow - high noise level). The slope of the line is a measure for the position gain of the subject. B. same as in A. but for the high control cost conditions (green - low noise level, red - high noise level). C.–F. Compares various measures between the high and low noise conditions. A risk-neutral controller predicts values to be the same for both condition (dashed line), a risk-averse controller predicts values to fall above the dashed line and a risk-seeking controller below it. C. Negative position gain for the high noise condition plotted against the low noise condition for all six subjects in the low control cost conditions (subject 5 in black, ellipses show the standard deviation). The dashed line represent equality between the gains. D. as C. but for the high control cost conditions. E. Negative velocity gain for the high noise condition plotted against the low noise condition for all six subjects for the low control cost conditions (ellipses show the standard deviation). F. as E. but for the high control cost conditions.</p

Simulations of an optimal controller with incomplete state observation and sensorimotor delay.

<p>A.–C. Changes in motor command with position for a fixed velocity () for the low noise level (green) and for the high noise level (red). D.–F. Contribution of control cost to total cost (control cost+error cost). A. & D. - Predictions of a risk-neutral controller. B. & E. - Predictions of a risk-averse controller. C. & F. - Predictions of a risk-preferring controller.</p

Contribution of control cost to total cost, and extra cost from risk sensitivity.

<p>A. Contribution of control cost to total cost for the high noise condition plotted against the low noise condition for the low control cost conditions (ellipses show 1 s.e.m. across all 250 trials). B. as A. but for the high cost level. C. Estimated extra cost in percent of a risk-sensitive controller with incomplete observation and sensorimotor delay based on the experimentally inferred -parameters for the low cost level. D. as C. but for the high cost level. E. Relationship between extra cost of a risk-sensitive controller relative to a risk-neutral controller for a range of -values overlaid with the subjects' experimentally inferred -parameters standard deviation for the low cost level. F. as E. but for the high cost level.</p

Analysis of possible confounds and sensorimotor delay.

<p>A. Results of the multilinear regression analysis of the first 15 trials for the low and the high noise condition. The subjects' data was pooled according to whether they began the experiment with a low gain or with a high gain (green and blue - low gain first; red and yellow - high gain first). B. Number of trials for different angles of the velocity vector of the ball with the wall upon impact (dark red - hypothetical impact angle of the ball had the subjects not intervened, dark blue - actual impact angle during the experiment). C. R-values of the multilinear regression analysis averaged across all subjects and conditions for different sensorimotor delays.</p

Schematic of the task.

<p>Subjects attempted to move a virtual ball (represented by the green circle) to the center of a target line (represented by the black horizontal line). The ball moved with constant y-velocity and hit the target after 1 s, whereas it moved with Brownian motion in the x-direction. Final positional errors were penalized by a quadratic cost function that was displayed as a parabola and the error cost was displayed at the end of the trial (blue bar). Subjects could exert control on the x position of the ball by moving their hand to the left or right (gray solid and dashed arrow lines). This incurred a control cost which was the quadratic in the control signal and the cumulative across a trial (yellow bar) was constantly displayed. At the end of the trial subjects received feedback of the total cost, the sum of control and error cost (yellow-blue bar). Subjects were required to minimize the total cost on average and were tested on four conditions (2 noise levels×2 control cost levels). The path taken by the ball is shown for a typical trial.</p

Predictions of optimal feedback control models.

<p>A risk-neutral optimal control model <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000857#pcbi.1000857-Todorov1" target="_blank">[4]</a>, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000857#pcbi.1000857-Todorov2" target="_blank">[17]</a> attempts to minimize the mean of the cost function. As a result, its policy (that is the motor command applied for a given state of the world) is independent of the noise variance N. In contrast, a risk-sensitive optimal control model <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000857#pcbi.1000857-Whittle1" target="_blank">[22]</a>, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000857#pcbi.1000857-Jacobson1" target="_blank">[34]</a> minimizes a weighted combination of the mean and variance of the cost. Additional variance is an added cost for a risk-averse controller (), whereas it makes a movement strategy more desirable for a risk-seeking controller (). As a consequence, the policy of the controller changes with the noise level N depending on its risk-attitude . A.–C. Changes in motor command with the state of the ball (its positional deviation from the center) for a low noise level (green) and for a high noise level (red) for the risk-neutral (A), risk-averse (B) and risk-seeking (C) controllers. The slope of the lines is equivalent to the control gain of the controller. D.–F. Contribution of control cost to total cost (control cost+error cost) for the risk-neutral (D), risk-averse (E) and risk-seeking (F) controllers.</p