Sequential Decisions: A Computational Comparison of Observational and Reinforcement Accounts

<div><p>Right brain damaged patients show impairments in sequential decision making tasks for which healthy people do not show any difficulty. We hypothesized that this difficulty could be due to the failure of right brain damage patients to develop well-matched models of the world. Our motivation is the idea that to navigate uncertainty, humans use models of the world to direct the decisions they make when interacting with their environment. The better the model is, the better their decisions are. To explore the model building and updating process in humans and the basis for impairment after brain injury, we used a computational model of non-stationary sequence learning. RELPH (Reinforcement and Entropy Learned Pruned Hypothesis space) was able to qualitatively and quantitatively reproduce the results of left and right brain damaged patient groups and healthy controls playing a sequential version of Rock, Paper, Scissors. Our results suggests that, in general, humans employ a sub-optimal reinforcement based learning method rather than an objectively better statistical learning approach, and that differences between right brain damaged and healthy control groups can be explained by different exploration policies, rather than qualitatively different learning mechanisms.</p></div

The mean (and standard deviation) of each parameter for RBDs vs. HCs.

<p>The mean (and standard deviation) of each parameter for RBDs vs. HCs.</p

Frequency of <i>n</i> (length of STM) for HCs and RBDs.

<p>Frequency of <i>n</i> (length of STM) for HCs and RBDs.</p

The average win rate for patient groups versus RELPH.

<p>Each plot represents the average win rate for (A) greedy-RELPH and (B) RELPH, (red lines) against the average win rate of (A) LBD, (B) RBD patients (blue lines) over the last 200 trials in the RPS experiment of Danckert et al. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0094308#pone.0094308-Danckert1" target="_blank">[13]</a>. Error bars represent the standard error of the mean.</p

The average win rate of HCs versus ELPH and RELPH.

<p>Each plot shows the average win rate over the last 200 trials in the RPS experiment of Danckert et al. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0094308#pone.0094308-Danckert1" target="_blank">[13]</a> for HCs versus (A) ELPH, (B) non-greedy ELPH and (C) RELPH. The blue line represents the average win rate of HCs. The red line shows the average win rate for the (A) ELPH, (B) non-greedy ELPH and (C) RELPH. Error bars represent the standard error of the mean.</p

AIC value and Bayes factor computed for each model (ELPH and RELPH) per each group separately.

<p>Bayes factor is calculated as 2ln(k) in which . D in this formula is the observed data which in our case is the participants' sequence of plays.</p