UKS test thresholds and associated p-value limits.

<p>For ten population sizes <i>I</i> from 5 to 30 individuals, the table indicates the Kolmogorov-Smirnov test threshold K_th for type I error rates equal to .05 (column 2) and .01 (column 5). Column 3 and 6 indicate the minimum number <i>n</i>_min of <i>p</i>-values required for the UKS test to be significant. These <i>p</i>-values have to be lower than the limit p_min indicated in columns 4 and 7. Note that the UKS test is significant as soon as <i>n</i>_min + <i>m p</i>-values are below p_min + <i>m</i>/<i>I</i> for any m between 0 and <i>I</i>-<i>n</i>_min. By construction, the limit for <i>I p</i>-values is equal to 1-K_th.</p

Robustness with skewed data.

<p>Rates of type I errors in UKS test for 3 representative experimental designs (lines) and the 4 skewed distributions shown in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0039059#pone-0039059-g003" target="_blank">Figure 3B</a> (columns). In each design, the UKS test was applied before and after log-transforming the random datasets. The rates of each design are equal to the percentages of 60000 random datasets with null factor effect that were found significant at the 0.05 threshold by the UKS test. The type I error rates obtained for the same data with Kruskal-Wallis test substituted to Anova are also indicated for the third design. Overall, either log-transformation of skewed data or use of a per-individual nonparametric test guards the UKS test against excessive type I errors.</p

Comparison of type II error rates in UKS test and RM Anovas

<p>. Results of a simulation study based on over one billion datasets. Each dataset represents the data of 10 individuals performing 10 trials in each of the 2 levels of a factor. Each data point was obtained by adding to the fixed central value of the level (−1/√2 or +1/√2) two random Gaussian values representing individual idiosyncrasies and trial-to-trial errors (see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0039059#s4" target="_blank">Methods</a>). <i>Panel A:</i> Median probability (Z-axis) yielded by RM Anovas as a function of the standard deviations of subject-factor interaction (X-axis, rightwards) and average of 10 trial-to-trial errors (Y-axis, leftwards). <i>Panel B:</i> Median probability yielded by the UKS test for the same random data. <i>Panel C:</i> superimposition of the surfaces displayed in panel A and B. Note that in conditions when UKS test is less powerful than ANOVA (larger median p), the difference in power is never dramatic; the converse is not true. <i>Panel D:</i> 2D-isolines of the surfaces in panel C for median probabilities. 001 (red), .01 (orange), .05 (green), .10 (light blue) and .20 (dark blue). Black line: projection of the intersection of the two surfaces; RM Anova is more powerful (smaller median probability) than the UKS test for points leftwards of the black line. Note that scaling the X-axis to the SD of within-level averages of trial-to-trial errors gives a symmetrical aspect to RM Anova surface and projection.</p

Type II errors and reproducibility with heterogeneous experimental effects

<p>. Each panel displays the proportion of significant hypothetical experiments as a function of the difference <i>d</i> between the constant values of experimental effect in 2 (panels A–E) or 3 sub-populations (panel F). The lines show the proportion of significant tests in 10000 hypothetical experiments for 41 values of <i>d</i> from 0 to 8 by .2 steps for RM Anovas (continuous line) and the UKS test at both the .05 (dashed line) and.01 threshold (dotted line). The gray part of lines indicates the 0.211–0.789 range of proportion of significant tests for which the probability that two subsequent experiments yield conflicting outcomes exceeds 1/3. Each experiment consists in 10 individuals performing 8 trials in a baseline condition and in an experimental condition. Trial errors are drawn from a Gaussian distribution with parameters 0 and √8, so that the average of the experimental condition has a Gaussian distribution centered on –<i>d</i>, 0 or +<i>d</i> (Insets) with unitary variance. The proportion and center of the subpopulations varied across studies. In the first study (<i>panel A</i>), the experimental effect was set to 0 for 10% of the population, and to <i>d</i> for the remaining 90%. In the other studies (<i>Panels B–F</i>), the effects and proportions were as follows: [0, 20%; <i>d</i>, 80%]; –<i>d</i>, 10%; <i>d</i>, 90%]; [0, 40%; <i>d</i>, 60%]; –<i>d</i>, 20%; <i>d</i>, 80%]; [–<i>d</i>, 10%; 0; 20%; <i>d</i>, 70%]. For each hypothetical experiment, the 10 individual effects were drawn with replacement from a set of –<i>d</i>, 0 and +<i>d</i> values in the above proportions (for <i>d</i> = 0, the proportion of significant tests is equal to the nominal type I error rate). We conclude that when factor effects vary across individuals as modeled by a mixture of Gaussians, UKS tests yield more reproducible outcomes than RM Anovas and have lower type II errors.</p

Robustness with violations of heteroscedasticity assumption.

<p>Rates of type I errors in repeated-measures Anovas and UKS test for 3 representative experimental designs (lines) and the same 4 degrees of heteroscedasticity as in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0039059#pone-0039059-g003" target="_blank">Figure 3A</a> (columns). Rates are averages of designs with 5, 10 and 20 individuals. The rates of each design are equal to the percentages of 60000 random datasets found significant at the 0.05 threshold as the effect of factor was set to zero. Bold values indicate large excess of type I errors. UKS (and RM Anova) are globally robust to violations of heteroscedasticity.</p

Violations of homoscedasticity and normality assumptions in one-way Anova design: compared robustness of RM Anova and UKS test.

<p><i>Panel A:</i> Violation of equal variance assumption. Curves display trial-to-trial errors distributions in the factor levels with the smallest and largest variance for the 4 degrees of heteroscedasticity investigated in simulation studies (see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0039059#s4" target="_blank">Methods</a>). The numbers under the curves indicate the average percentage of type I errors (false positives) for RM Anovas, individual Anovas and the UKS test procedure, respectively. Numbers above 5% indicate an excess of significant datasets with respect to the tests threshold (0.05). We observe that the UKS test, as the RM Anova, is robust to heterogeneity of variance. <i>Panel B:</i> Violation of normality assumption. Curves display the empirical distributions of trial-to-trial errors drawn from the following 4 distributions: gamma with k = 4; lognormal with μ = 0 and σ = 1/√2; Weibull with k = 1.2 and λ = 0.5; exponential with λ = 0.4 (see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0039059#s4" target="_blank">Methods</a>). Boxes: Normal probability plots of typical residuals from an Anova applied to skewed data randomly drawn from the above distribution. For the displayed residuals (10 individuals × 3 levels × 10 repetitions with a median coefficient of correlation r), skewness is significant at the .01 threshold when r <0.9942. The numbers under the boxes indicate the across-designs average percentage of type I errors (false positives) for individual Anovas and UKS test applied to raw data or after a logarithmic transformation. Numbers above 5% indicate an excess of significant datasets with respect to the threshold used (0.05). When data is skewed, the UKS test should be used in conjunction with individual nonparametric tests (see text, Part 7), or data should be (log-)transformed.</p

Experimental setup and behavioral task.

<p><b>A.</b> Experimental setup. The animal was seated in front of a 152 x 114 cm screen on which a computer-generated scene was projected in stereo. The animal was equipped with shutter glasses synchronized with the projection and could move in the virtual world via a joystick. A juice dispenser delivered reward directly in the animal’s mouth when the monkey reached a hidden rewarded area. <b>B.</b> View from above of the star maze. Five landmarks were placed between the five arms of the maze at a radius twice the arms’ length. A reward was given to the animal when he reached the end of an arbitrarily chosen arm (in this case, the arm between the sunflower and the house). <b>C.</b> A sequential illustration of the animal’s position and field of view at key representative events of a trial. (1) The animal starts at one end of a path and moves towards the center, (2) turns left or right in the center, (3) chooses one path, (4) enters the chosen path and is rewarded at the end if correct, and (5) the animal is relocated (joystick disengaged) to the next start. <b>D.</b> First-person view of the five events described in <b>C</b>, with a heat map of the monkey’s gaze fixations overlaid on the scene illustrating the animal’s scanning interests. Arrows indicate the main direction of motion of the animal. <b>E.</b> Illustration of the steps described in <b>C</b> and <b>D</b> in the actual maze space. Monkey’s moves are represented by colored arrows. <b>F.</b> Illustration of the state space in which neuronal data was analyzed. The same steps as in <b>E</b> are plotted in the state-space graph with corresponding colors. For convenience, the animal’s current position in the graph also denotes the animal’s current straight ahead direction. For example, a position in the northeastern part of the graph corresponds to the animal viewing the northeast from its physical position. The state-space representation parses the animal’s trajectories into a series of action- or position-triggered transitions between choice points (graph vertices). Starting positions are figured as black dots. All actions that can eventually lead to the reward are in solid lines, while dashed lines indicate either erroneous actions leading to the end of unrewarded arms (open circles) or the path to the next start, outside the maze arms. This representation allows describing in a similar way the moves that include a translation and the purely rotational moves made in the center of the maze (expanded inset in the black square). Rotations of 72° (angle between two maze arms) are mapped to the central part of the graph, with counterclockwise rotations innermost (e.g., in red). Rotations of 36° (angle between landmark and maze arm) are mapped to the outer circular arcs (either clockwise or counterclockwise; e.g., in cyan). <b>G.</b> Mapping the animal’s 3-D point of regard. Left: three-dimensional schematic of the maze (green), monkey (brown), and point of gaze (red dot). Blue rectangles represent the location of the landmarks. For ease of representation, we define an invisible cylindrical wall running through the landmark centroids. Right: convention for the flattened representation of the point-of-gaze map. When directed further than the distance to the landmark wall, the point of gaze was computed as directed towards this wall; then, in a second step, this wall was flattened as an annulus to create the final 2-D map. <b>H.</b> Heat map of the point of gaze, overlaid on a schematic of the maze for one session (monkey S). The regions of interest explored by the animal are the ends of the paths, the landmarks, and the rewarded area.</p

Sensitivity to landmark identity and relative distance.

<p><b>Left panel</b>. Activity of a cell for each of the four landmarks viewed at four intervals of relative distances on the entry path (RD1 to RD4, see <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2001045#sec017" target="_blank">Materials and Methods</a>). Top row: schema of the maze with these distance intervals illustrated as areas for each landmark (southwestern landmark in blue, northwestern in red, northeastern in green, and southeastern in black). The pictures above the rasters show a still image of the monkey’s view of the landmark at each relative distance symbolized by dotted lines on the raster (12, 8, 4), the last one being at 0. Each raster represents the activity of the cell to each landmark as the animal moves forward in the corresponding path. On these rasters, each line is a trial. The bottom graph shows the average cell activity over all trials. <b>Right panel</b>. Activity of a different cell recorded during the same session as the cell shown in <b>the Left panel</b>. Underlying data can be found at <a href="http://dx.doi.org/10.6080/K0R49NQV" target="_blank">http://dx.doi.org/10.6080/K0R49NQV</a>.</p

State-space selectivity.

<p><b>A.</b> Schematics illustrating the analysis method whereby activities corresponding to the same location (maze center) and direction (dashed sector) were compared on the different state-space transitions (in red). <b>B</b>. Histogram of the state-space selectivity indices across the responsive cells. Cells significantly invariant to current transition are in blue; cells significantly context-dependent are in red (permutation tests). The distribution of indices for the surrogate spike sets is shown in dashed green. <b>C–D</b>. State-space maps (restricted to the center) of two context-dependent cells. Underlying data can be found at <a href="http://dx.doi.org/10.6080/K0R49NQV" target="_blank">http://dx.doi.org/10.6080/K0R49NQV</a>.</p

Landmark viewpoint-invariant versus viewpoint-dependent cells.

<p>Top row: schematics of the monkey’s five different viewpoints for either the landmark immediately left or the landmark immediately right from the reward location. For every path, the landmark appears either on the animal’s left or right. <b>A–C</b>. Individual examples of cell activity (average and trial-by-trial raster histogram; cells numbered as in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2001045#pbio.2001045.g002" target="_blank">Fig 2</a>) aligned on the landmark’s left or right entries in the animal’s field of view. The color codes correspond to the activity on the individual paths identified in the top row. Cells displayed in <b>A</b> and <b>C</b> discriminate between different viewpoints, while the cell displayed in <b>B</b> does not. <b>D.</b> Path selectivity index calculated for the different viewpoints of the landmark left or right of the reward (best landmark for each cell). In red are cells for which the index was significantly higher from chance (viewpoint dependent), and in blue are cells for which the index was significantly lower than chance (viewpoint invariant). The distribution of indices for the surrogate spike sets is shown in dashed green. Underlying data can be found at <a href="http://dx.doi.org/10.6080/K0R49NQV" target="_blank">http://dx.doi.org/10.6080/K0R49NQV</a>.</p