Uncertainty in action-value estimation affects both action choice and learning rate of the choice behaviors of rats

The estimation of reward outcomes for action candidates is essential for decision making. In this study, we examined whether and how the uncertainty in reward outcome estimation affects the action choice and learning rate. We designed a choice task in which rats selected either the left-poking or right-poking hole and received a reward of a food pellet stochastically. The reward probabilities of the left and right holes were chosen from six settings (high, 100% vs. 66%; mid, 66% vs. 33%; low, 33% vs. 0% for the left vs. right holes, and the opposites) in every 20–549 trials. We used Bayesian Q-learning models to estimate the time course of the probability distribution of action values and tested if they better explain the behaviors of rats than standard Q-learning models that estimate only the mean of action values. Model comparison by cross-validation revealed that a Bayesian Q-learning model with an asymmetric update for reward and non-reward outcomes fit the choice time course of the rats best. In the action-choice equation of the Bayesian Q-learning model, the estimated coefficient for the variance of action value was positive, meaning that rats were uncertainty seeking. Further analysis of the Bayesian Q-learning model suggested that the uncertainty facilitated the effective learning rate. These results suggest that the rats consider uncertainty in action-value estimation and that they have an uncertainty-seeking action policy and uncertainty-dependent modulation of the effective learning rate

Pre-attentive, context-specific representation of fear memory in the auditory cortex of rat.

Neural representation in the auditory cortex is rapidly modulated by both top-down attention and bottom-up stimulus properties, in order to improve perception in a given context. Learning-induced, pre-attentive, map plasticity has been also studied in the anesthetized cortex; however, little attention has been paid to rapid, context-dependent modulation. We hypothesize that context-specific learning leads to pre-attentively modulated, multiplex representation in the auditory cortex. Here, we investigate map plasticity in the auditory cortices of anesthetized rats conditioned in a context-dependent manner, such that a conditioned stimulus (CS) of a 20-kHz tone and an unconditioned stimulus (US) of a mild electrical shock were associated only under a noisy auditory context, but not in silence. After the conditioning, although no distinct plasticity was found in the tonotopic map, tone-evoked responses were more noise-resistive than pre-conditioning. Yet, the conditioned group showed a reduced spread of activation to each tone with noise, but not with silence, associated with a sharpening of frequency tuning. The encoding accuracy index of neurons showed that conditioning deteriorated the accuracy of tone-frequency representations in noisy condition at off-CS regions, but not at CS regions, suggesting that arbitrary tones around the frequency of the CS were more likely perceived as the CS in a specific context, where CS was associated with US. These results together demonstrate that learning-induced plasticity in the auditory cortex occurs in a context-dependent manner

Stable sound decoding despite modulated sound representation in the auditory cortex

Two-photon calcium imaging data (df/f) from the mouse auditory cortex during a two-alternative choice auditory decision-making task. This is the data set of the paper titled: "Stable sound decoding despite modulated sound representation in the auditory cortex" Current Biology 33, 1–14, October 23, 2023 DOI:https://doi.org/10.1016/j.cub.2023.09.03

Localized and global representation of prior value, sensory evidence, and choice in male mouse cerebral cortex

Abstract Adaptive behavior requires integrating prior knowledge of action outcomes and sensory evidence for making decisions while maintaining prior knowledge for future actions. As outcome- and sensory-based decisions are often tested separately, it is unclear how these processes are integrated in the brain. In a tone frequency discrimination task with two sound durations and asymmetric reward blocks, we found that neurons in the medial prefrontal cortex of male mice represented the additive combination of prior reward expectations and choices. The sensory inputs and choices were selectively decoded from the auditory cortex irrespective of reward priors and the secondary motor cortex, respectively, suggesting localized computations of task variables are required within single trials. In contrast, all the recorded regions represented prior values that needed to be maintained across trials. We propose localized and global computations of task variables in different time scales in the cerebral cortex

Frequency tuning properties in a population of neural activities.

<p>(A) Cortical recruitment functions (CRFs) in silence (i) and noise (ii) in all of the auditory fields. Percentages of recording sites active are given as a function of the test frequency-intensity pair. Test frequencies are combined into 6 groups, each with a 1 octave frequency (i.e., 3 test frequencies). (B) CRFs at 70 dB SPL. Percentages of recording sites active to 70-dB-SPL tones are shown as a function of test frequencies. Asterisks indicate significant differences between the naïve and conditioned groups (z-test after Bonferroni correction for 6 comparisons: *, p<0.05; **, p<0.01). (C) Bandwidths of frequency response areas at 70 dB SPL as a function of the best frequency (BF) of recording site. Asterisks indicate the significance of post-hoc analyses (Mann-Whitney U-test after Bonferroni correction for 6 comparisons: **, p<0.01). (D) CRFs at 70 dB SPL in indicated fields. Asterisks indicate significant differences between the naïve and conditioned groups (Z-test after Bonferroni correction: *, p<0.05). (E) Bandwidths in indicated fields. Asterisks indicate the significance of post-hoc analyses (Mann-Whitney U-test after Bonferroni correction: *, p<0.05).</p

Context-dependent auditory fear conditioning.

<p>(A) Procedure used for conditioning (i) and measurement (ii). (i) In the conditioning session, 4 silent and 3 noise conditions were prepared, and were sequentially alternated. In a silent-condition block, only a conditioned stimulus (CS) consisting of a 20-kHz tone was presented 10 times. In a noise-condition block, a white noise stimulus was continuously presented during the block, and an unconditioned stimulus (US) consisting of an electrical foot shock was associated with the CS 5 times. The total time of the conditioning session was approximately 2.5 h. (ii) The measurement of freezing time of rats was conducted the day after the conditioning session. CS only, noise only, and CS under noise were presented in order each for 3 min. (B) Freezing times of rats. Asterisks indicate the significance of post-hoc analyses (Mann-Whitney U-test: **, p<0.01).</p

Breakdown of tone-responsive sites.

<p>Breakdown of tone-responsive sites.</p

Characterization of multi-unit activities.

<p>(A) Representative frequency response area (FRA) in the auditory cortex in a naïve (i) and a conditioned rat (ii). The FRAs were different under silent and noise conditions. Spike rates are shown in gray scale for a given pair of test frequencies (abscissa) and intensities (ordinate). The insets at lower left show action potential waveforms. Scale bar: vertical axis, 50 μV; horizontal axis, 0.5 ms. (B) Histogram of PSTH peak latency in silence (black) and under the noise condition (gray).</p

Accuracy of frequency representations at 70 dB SPL.

<p>(A) Population average of frequency-representation accuracy as a function of test frequency in silence (left) and noise (right) conditions. The test frequencies are categorized into 6 groups, with a bin width of 1 octave (i.e., 3 test frequencies). Data are presented as means and standard errors. Daggers show significances in the interaction term in two-way ANOVA here and hereafter (†, p<0.05). Asterisks indicate the significance of post-hoc analyses (Mann-Whitney U-test after Bonferroni correction for 6 comparisons, **, p<0.01). (B) Breakdown list of frequency-representation accuracy as a function of the best frequency (BF) of recording sites. The accuracies in (A) are broken down according to the BF of each recording site (ordinate), and shown by color scale. BF and the test frequencies are binned with 1 octave intervals. The diagonal usually had higher accuracies than others, supporting the notion that neurons accurately represented a tone with their own BF. White triangles indicate that the accuracies in silence and noise were significantly different within either the naïve (i) or conditioned group (ii). Blue triangles indicate that accuracies in silence (left) were significantly different between the naïve and conditioned groups, while red triangles indicate significant differences under noise (right). The orientation of the triangle shows the increase () or decrease () in the accuracy. For example, a white in the naïve group in the silent condition (left column of (i)) indicates that the accuracy in silence was higher than under noise. A red in the conditioned group under the noise condition (right column of (ii)) indicates that, under noise, the accuracy in the conditioned group was lower than that in the naïve group (post-hoc Mann-Whitney U-test after Bonferroni correction for 6 comparisons, p<0.05). (C) Population average of frequency representation accuracies in indicated fields. Asterisks indicate the significance of post-hoc analyses (Mann-Whitney U-test after Bonferroni correction for 6 comparisons: *, p<0.05; **, p<0.01).</p