Model simulation of predicted reward values for Gabor patch stimuli.

<p>(a) Changes of <i>Vr</i>(<i>t</i>) and (b) changes in percentage of trials in which <i>Vr</i>(<i>t</i>) was higher than <i>Vn</i>(<i>t</i>). Results obtained using three representative values of <i>α</i> (0.01, 0.05, and 0.1) are depicted. <i>Vr</i>(<i>t</i>) decreased faster in the WITHOUT model than WITH model. Error bars represent ±1 s.e.m.</p

Model simulation of reward prediction error.

<p>Changes in <i>δ</i>(<i>t</i>) for trials in which the unpredicted reward was delivered (positive prediction error trials: subjects judged orientation conditioned with tasteless saliva but juice was delivered). The vertical axis represents the average of <i>δ</i>(<i>t</i>) for each condition in each session. Results for <i>α</i> = 0.01, 0.05 and 0.1 are depicted. Solid black lines represent the WITH model whereas dotted gray lines represent the WITHOUT model. Black squares represent high-contrast (90% correctness) trials whereas open circles represent low-contrast trials (60% correctness). <i>δ</i>(<i>t</i>) for high and low-contrast stimuli were almost identical in the WITHOUT model, but differed in the WITH model for all learning rates.</p

How Sound Symbolism Is Processed in the Brain: A Study on Japanese Mimetic Words

<div><p>Sound symbolism is the systematic and non-arbitrary link between word and meaning. Although a number of behavioral studies demonstrate that both children and adults are universally sensitive to sound symbolism in mimetic words, the neural mechanisms underlying this phenomenon have not yet been extensively investigated. The present study used functional magnetic resonance imaging to investigate how Japanese mimetic words are processed in the brain. In Experiment 1, we compared processing for motion mimetic words with that for non-sound symbolic motion verbs and adverbs. Mimetic words uniquely activated the right posterior superior temporal sulcus (STS). In Experiment 2, we further examined the generalizability of the findings from Experiment 1 by testing another domain: shape mimetics. Our results show that the right posterior STS was active when subjects processed both motion and shape mimetic words, thus suggesting that this area may be the primary structure for processing sound symbolism. Increased activity in the right posterior STS may also reflect how sound symbolic words function as both linguistic and non-linguistic iconic symbols.</p></div

Brain regions showing significant correlation between fMRI signals and reward prediction error values at the time of reward delivery (<i>n</i> = 23).

<p>A white arrow indicates the midbrain region showing a significant correlation (yellow— <i>P</i><.05, corrected for false discovery rate: red— <i>P</i><.001, uncorrected for multiple comparison) with the variation of prediction error <i>δ</i>(<i>t</i>) at the time of reward delivery calculated using the WITH model using <i>α</i> = 0.05.</p

Regions with BOLD responses correlated with predicted reward values <i>V</i>(<i>t</i>) at the time of presentation of Gabor patch stimuli (<i>α</i> = 0.05).

<p>Only foci with cluster size >5 are reported.</p><p>*, <i>P</i><.05, FDR correction. Other areas were significant at <i>P</i><.001, uncorrected for multiple comparison. The rightmost column shows statistical significance between the WITH model and the WITHOUT model (two-tailed paired <i>t</i>-test). WITH>WITHOUT, significantly higher effect size in the WITH model than in the WITHOUT model. WITHOUT>WITH, significantly higher effect size in the WITHOUT model than in the WITH model.</p

Effect size at the peak midbrain voxel in each learning rate.

<p>The effect sizes at the peak midbrain voxel across the eight learning rates are shown for each model (filled squares and thick lines for the WITH model, open circles and dotted lines for the WITHOUT model). Vertical axis represents the effect size (parameter estimates for the regressor of the reward prediction error <i>δ</i>(<i>t</i>)) at the peak voxel averaged for 23 subjects based on the data up to session 3. The effect size was significantly greater for the WITH model than for the WITHOUT model in most learning rates (two-tailed paired <i>t</i>-test: *, <i>p</i><.05; **, <i>p</i><.01). Error bars represent ±1 s.e.m.</p

Study paradigm for Experiment 2.

<p>Experiment 2 used an event-related design. Stimuli were presented in the following order: 5-sec fixation point, 1-word instruction (presented either for 2.5, 3.0, or 3.5 sec) indicating the trial type ( “shape” or “motion”), 5-sec video clip, 3-sec fixation point, and a mimetic word. Video clips depicted an agent who stayed still in the shape trials and moved from left to right in the motion trials. During the presentation of a mimetic word, participants pressed a button to indicate the degree of match between the referent and the mimetic word. The mimetic word depicted in this example is (<i>hyoihyoi</i>) which means “jumping effortlessly” in this context.</p