Reinforcement Learning With Parsimonious Computation and a Forgetting Process

Decision-making is assumed to be supported by model-free and model-based systems: the model-free system is based purely on experience, while the model-based system uses a cognitive map of the environment and is more accurate. The recently developed multistep decision-making task and its computational model can dissociate the contributions of the two systems and have been used widely. This study used this task and model to understand our value-based learning process and tested alternative algorithms for the model-free and model-based learning systems. The task used in this study had a deterministic transition structure, and the degree of use of this structure in learning is estimated as the relative contribution of the model-based system to choices. We obtained data from 29 participants and fitted them with various computational models that differ in the model-free and model-based assumptions. The results of model comparison and parameter estimation showed that the participants update the value of action sequences and not each action. Additionally, the model fit was improved substantially by assuming that the learning mechanism includes a forgetting process, where the values of unselected options change to a certain default value over time. We also examined the relationships between the estimated parameters and psychopathology and other traits measured by self-reported questionnaires, and the results suggested that the difference in model assumptions can change the conclusion. In particular, inclusion of the forgetting process in the computational models had a strong impact on estimation of the weighting parameter of the model-free and model-based systems

Effects of interoceptive accuracy on timing control in the synchronization tapping task

Humans often perform rhythmic synchronized movements. Professional musicians and dancers particularly perform such movement tasks well and have a higher interoceptive accuracy (IAcc) than non-musicians and non-dancers. We thus hypothesized that rhythmic synchronized movements might be enhanced by a higher IAcc. To investigate this hypothesis, this study conducted a heartbeat counting task and a rhythmic synchronization tapping task with normal (easier) and slow (harder) tempi metronomes. Inconsistent with our hypothesis, however, a higher IAcc was negatively correlated with timing control, but only in the slow tempo condition [r (30) = 0.46, p < 0.05]. This suggests that a higher IAcc did not enhance timing control in rhythmic synchronized movements but rather weakened it, resting heart rate variability was not correlated with timing control

Complex sequencing rules of birdsong can be explained by simple hidden Markov processes

Complex sequencing rules observed in birdsongs provide an opportunity to investigate the neural mechanism for generating complex sequential behaviors. To relate the findings from studying birdsongs to other sequential behaviors, it is crucial to characterize the statistical properties of the sequencing rules in birdsongs. However, the properties of the sequencing rules in birdsongs have not yet been fully addressed. In this study, we investigate the statistical propertiesof the complex birdsong of the Bengalese finch (Lonchura striata var. domestica). Based on manual-annotated syllable sequences, we first show that there are significant higher-order context dependencies in Bengalese finch songs, that is, which syllable appears next depends on more than one previous syllable. This property is shared with other complex sequential behaviors. We then analyze acoustic features of the song and show that higher-order context dependencies can be explained using first-order hidden state transition dynamics with redundant hidden states. This model corresponds to hidden Markov models (HMMs), well known statistical models with a large range of application for time series modeling. The song annotation with these models with first-order hidden state dynamics agreed well with manual annotation, the score was comparable to that of a second-order HMM, and surpassed the zeroth-order model (the Gaussian mixture model (GMM)), which does not use context information. Our results imply that the hierarchical representation with hidden state dynamics may underlie the neural implementation for generating complex sequences with higher-order dependencies

Consolidated bioprocessing of corn cob-derived hemicellulose: engineered industrial Saccharomyces cerevisiae as efficient whole cell biocatalysts

Background Consolidated bioprocessing, which combines saccharolytic and fermentative abilities in a single microorganism, is receiving increased attention to decrease environmental and economic costs in lignocellulosic biorefineries. Nevertheless, the economic viability of lignocellulosic ethanol is also dependent of an efficient utilization of the hemicellulosic fraction, which contains xylose as a major component in concentrations that can reach up to 40% of the total biomass in hardwoods and agricultural residues. This major bottleneck is mainly due to the necessity of chemical/enzymatic treatments to hydrolyze hemicellulose into fermentable sugars and to the fact that xylose is not readily consumed by Saccharomyces cerevisiaethe most used organism for large-scale ethanol production. In this work, industrial S. cerevisiae strains, presenting robust traits such as thermotolerance and improved resistance to inhibitors, were evaluated as hosts for the cell-surface display of hemicellulolytic enzymes and optimized xylose assimilation, aiming at the development of whole-cell biocatalysts for consolidated bioprocessing of corn cob-derived hemicellulose. Results These modifications allowed the direct production of ethanol from non-detoxified hemicellulosic liquor obtained by hydrothermal pretreatment of corn cob, reaching an ethanol titer of 11.1 g/L corresponding to a yield of 0.328 g/g of potential xylose and glucose, without the need for external hydrolytic catalysts. Also, consolidated bioprocessing of pretreated corn cob was found to be more efficient for hemicellulosic ethanol production than simultaneous saccharification and fermentation with addition of commercial hemicellulases. Conclusions These results show the potential of industrial S. cerevisiae strains for the design of whole-cell biocatalysts and paves the way for the development of more efficient consolidated bioprocesses for lignocellulosic biomass valorization, further decreasing environmental and economic costs.This work has been carried out at the Biomass and Bioenergy Research Infrastructure (BBRI)-LISBOA-01-0145-FEDER-022059, supported by Operational Programme for Competitiveness and Internationalization (PORTUGAL2020), by Lisbon Portugal Regional Operational Programme (Lisboa 2020) and by North Portugal Regional Operational Programme (Norte 2020) under the Portugal 2020 Partnership Agreement, through the European Regional Development Fund (ERDF) and has been supported by the Portuguese Foundation for Science and Technology (FCT) under the scope of the strategic funding of UIDB/04469/2020, the “Contrato-Programa” UIDB/04050/2020, the MIT-Portugal Program (Ph.D. Grant PD/BD/128247/2016 to Joana T. Cunha) and through Project FatVal (POCI-01-0145-FEDER-032506) and BioTecNorte operation (NORTE-01-0145-FEDER-000004) funded by the European Regional Development Fund under the scope of Norte2020 - Programa Operacional Regional do Norte.info:eu-repo/semantics/publishedVersio

Reliability of computational models

R scripts to illustrate the effect of prior on test-retest reliability of the computational model parameters