Rapid reaching movements in tasks with competing targets.

<p>Top row illustrates experimental results in rapid reaching tasks with multiple potential targets [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004402#pcbi.1004402.ref012" target="_blank">12</a>, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004402#pcbi.1004402.ref022" target="_blank">22</a>, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004402#pcbi.1004402.ref023" target="_blank">23</a>] (images are reproduced with permission of the authors). When the target position is known prior to movement onset, reaches are made directly to that target (black and green traces in <b>A</b>), otherwise, reaches aim to an intermediate location, before correcting in-flight to the cued target (red and blue traces in <b>A</b>). The competition between the two reaching policies that results in spatial averaging movements, is biased by the spatial distribution of the targets (<b>B</b>), by recent trial history (<b>C</b>) and the number of targets presented in each visual field (<b>D</b>). The bottom row (<b>E-H</b>) illustrates the simulated reaching movements generated in tasks with multiple potential targets. Each bottom panel corresponds to the reaching condition described on the top panels.</p

Sequential movements.

<p><b>A</b>: Examples of simulated trajectories for continuously copying a pentagon. <b>B</b>: Time course of the relative desirability values of the 5 individual policies (i.e., 5 segments) in a successful trial for copying a pentagon. The line colors correspond to the segments of the pentagon as shown in the top panel. The shape was copied counterclockwise (as indicated by the arrow) starting from the gray vertex. Each of the horizontal discontinuous lines indicate the completion time of copying the current segment. Notice that the desirability of the current segment peaks immediately after the start of drawing that segment and falls down gradually, whereas the desirability of the following segment starts rising while copying the current segment. Because of that, the consecutive segments compete for action selection frequently producing error trials, as illustrated in panel <b>C</b>. Finally, the panels (<b>D</b>) and (<b>E</b>) depict examples of simulated trajectories for continuously copying an equilateral triangle and a square, respectively, counterclockwise starting from the bottom right vertex.</p

Saccadic movements in tasks with competing targets.

<p><b>A</b>: Simulated saccadic movements for pair of targets with 30° (gray traces) and 90° (black traces) target separation. <b>B</b>: A method followed to visualize the relative desirability function of two competing saccadic policies (see <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004402#sec006" target="_blank">results</a> section for more details). <b>C</b>: Heat map of the relative desirability function at different states to saccade to the left target, at a 30° target separation. Red and blue regions corresponds to high and low desirability states, respectively. Black traces correspond to averaged trajectories in single-target trials. Notice the strong competition between the two saccadic policies (greenish areas). <b>D</b>: Similar to panel <i>C</i>, but for 90° target separation. In this case, targets are located in areas with no competition between the two policies (red and blue regions). <b>E</b>: Examples of saccadic movements (left column) with the corresponding time course of the relative desirability of the two policies (right column). The first two rows illustrate characteristic examples from 30° target separation, in which competition results primarily in saccade averaging (top panel) and less frequently in correct movements (middle panel). The bottom row shows a characteristic example from 90° target separation, in which the competition is resolved almost immediately after saccadic onset, producing almost no errors. <b>F</b>: Percentage of simulated averaging saccades for different degrees of target separation (red line)—green, blue and cyan lines describe the percentage of averaging saccades performed by 3 monkeys [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004402#pcbi.1004402.ref024" target="_blank">24</a>].</p

Encoding the order of policies in sequential movements.

<p><b>A</b>: Probability distribution of time to arrive at vertex <i>j</i> starting from the original state at time <i>t</i> = 0 and visiting all the precedent vertices. Each color codes the segments and the vertices of the pentagon as shown in the right inset. The pentagon is copied counterclockwise (as indicated by the arrow) starting from the purple vertex at <i>t</i> = 0. The gray trajectories illustrate examples from the 100 reaches generated to estimate the probability distribution of time to arrive at vertex <i>k</i> given that we started from vertex <i>k</i> − 1, <math><mrow><mi>P</mi><mo>(</mo><msubsup><mi>τ</mi><mrow><mi>a</mi><mi>r</mi><mi>r</mi><mi>i</mi><mi>v</mi><mi>e</mi></mrow><mi>k</mi></msubsup><mo>|</mo><msubsup><mi>τ</mi><mrow><mi>a</mi><mi>r</mi><mi>r</mi><mi>i</mi><mi>v</mi><mi>e</mi></mrow><mrow><mi>k</mi><mo>-</mo><mn>1</mn></mrow></msubsup><mo>)</mo></mrow></math>. <b>B</b>: Probability distribution <i>P</i>(<i>vertex</i> = <i>j</i>|<b>x</b>_<i>t</i>), which describes the probability to copy the segment defined by the two successive vertices <i>j</i> − 1 and <i>j</i> at state <b>x</b>_<i>t</i>. This probability distribution is estimated at time <i>t</i> = 0 and when arriving at the next vertex, we condition on completion, and <i>P</i>(<i>vertex</i> = <i>j</i>|<b>x</b>_<i>t</i>) is re-evaluated for the next vertices.</p

The architectural organization of the theory.

<p>It consists of multiple stochastic optimal control schemes where each of them is attached to a particular goal presented currently in the field. We illustrate the architecture of the theory using the hypothetical scenario of the soccer game, in which the player who is possessing the ball is presented with 3 alternative options—i.e., 3 teammates—located at different distances from the current state <b>x</b>_<i>t</i>. In such a situation, the control schemes related to these options are triggered and generate 3 action plans (<b>u</b>₁ = <i>π</i>₁(<b>x</b>_<i>t</i>), <b>u</b>₂ = <i>π</i>₂(<b>x</b>_<i>t</i>) and <b>u</b>₃ = <i>π</i>₃(<b>x</b>_<i>t</i>)) to pursue each of the individual options. At each time <i>t</i>, desirabilities of the each policy in terms of action cost and good value are computed separately, then combined into an overall desirability. The action cost of each policy is the cost-to-go of the remaining actions that would occur if the policy were followed from the current state <b>x</b>_<i>t</i> to the target. These action costs are converted into a relative desirability that characterizes the probability that implementing this policy will have the lowest cost relative to the alternative policies. Similarly, the good value attached to each policy is evaluated in the goods-space and is converted into a relative desirability that characterizes the probability that implementing that policy (i.e., select the goal <i>i</i>) will result in highest reward compare to the alternative options, from the current state <b>x</b>_<i>t</i>. These two desirabilities are combined to give what we call “relative-desirability” value, which reflects the degree to which the individual policy <i>π</i>_<i>i</i> is desirable to follow, at the given time and state, with respect to the other available policies. The overall policy that the player follows is a time-varying weighted mixture of the individual policies using the desirability value as weighted factor. Because relative desirability is time- and state- dependent, the weighted mixture of policies produces a range of behavior from “winner-take-all” (i.e., pass the ball) to “spatial averaging” (i.e., keep the ball and delay your decision).</p

Task, model, and inference.

<p><b>A. No-haptic condition.</b> The schematic shows a scene that contains a ball at some distance, and an observer who monocularly views a projected image of the ball (eye on left of image plane). In the absence of size information, the object's distance is ambiguous; e.g. the ball may be small and near (), medium-sized and mid-range (), large and far (), or anywhere in between, but still project to the same . The lower-right inset is the no-haptic condition Bayes' net that shows the generative direction (black arrows), and information flow during inference (dotted arrows). and both influence (black arrows), the likelihood of given is the plot labeled “” on the left. Inferring the ball's distance means propagating prior information about (“” plot on top-right) and to form a posterior over (labeled “”). Notice that regardless of the true (i.e. , , , black vertical lines in posterior plot), the posterior over is the same, and is often positioned quite far from the true . <b>B. Haptic condition.</b> The observer monocularly views an image of the scene and touches the ball beforehand to receive haptic size information, . Though the image only constrains possible values to those consistent with , because varies with it constrains more and can disambiguate . The lower-right inset is the haptic condition Bayes' net that shows the generative direction (black arrows), and the information flow (dotted arrows). The and both influence (black arrows), again the likelihood of given is the plot labeled “” on the left, but now the marginal posterior of given (plot labeled “”) captures information about . Inferring the ball's distance means propagating information, prior information about , and to form a posterior over (labeled “”). Notice that now different (i.e. , , , black vertical lines in ) induce different posterior distributions (different curves in ), and each is positioned much nearer to the respective true .</p

Perceptual model.

<p>Posterior distributions over given sensory input are depicted. The true values of and are 6.0 and 3.0, respectively. The red curves are when the haptic information is not incorporated, the blue curves are when it is incorporated. The curves on the bottom are the joint distributions marginalized over , to yield marginal posteriors over . The dotted vertical lines are the posterior means. The black dot is the true values, the purple dot is the prior mean. The top row is an observer who uses weak priors (high and ) and the bottom row is an observer who uses accurate priors (lower and ). The left column is an observer with accurate knowledge of the haptic noise () and the right column is an observer with inaccurate knowledge (overestimated) of haptic noise (). Notice that by using haptic information, the mean of the posterior becomes more accurate and the variance decreases. When prior information used, bias is introduced in that the means become less accurate, however the posterior variance decreases. Also notice that when the haptic cue noise is inaccurate the observer's posterior shifts toward the no-haptic integration observer.</p

Decision-making model.

<p>The left column shows model observers' response distributions given input and values 6.0 and 3.0, respectively. The top row is when accurate haptic noise knowledge is used, the bottom row is when inaccurate haptic noise is used. The solid vertical lines are the true values of . The solid distribution is an observer that uses the MAP estimate to make a distance judgment, the dashed distribution is an observer that draws posterior samples and averages their values, the dotted distribution is an observer that draws sample. The right column shows the corresponding SDs of the distributions in the left column. Notice that the sampling observers have less precise response distributions than the MAP-responder, and averaging over fewer samples yields less precise responses.</p

Experimental stimulus screenshot.

<p>The overlaid lines were not visible to the experimental participant, but depict various task elements; they are not drawn from the participant's viewpoint, but rather from a viewpoint elevated above the observer's head so they can be distinguished from each other the participant's viewpoint intersected the constraint line). They are: the constraint line (yellow dotted line), the ball's true movement path (green solid arrow), and ambiguous movement paths in the no-haptic condition (blue dotted arrows). The point at which the green arrow intersects the constraint line is the crossing distance. The points at which the blue arrows intersect the constraint line represent distance misjudgments. The participant's hand position was indicated by a 3 mm diameter blue sphere.</p

Example tuning curves, responses, and spike-count covariance matrix for the virtual cortical population.

<p>A. Example neural tuning curves for neurons with different BFs in primary auditory cortex. B. Example population response. The stimulus was a pure tone with a frequency of 1000: mean spike rates as a function of BF. Circles: simulated spike counts. C. Spike-rate covariance matrix. Entries on the diagonal correspond to spike-count variances for individual units and are equal to the spike rates shown in panel B. Off-diagonal entries correspond to covariances between the spike counts of different units, and are equal to the geometric mean of the units' spike rates times the spike-count correlation coefficient. D. Correlation matrix corresponding to the covariance matrix shown in panel C. See <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003336#s4" target="_blank">Methods</a> for details.</p