Protection from uncertainty in the exploration/exploitation trade-off

The exploration/exploitation trade-off (EE trade-off) describes how, when faced with several competing alternatives, decision-makers must often choose between a known good alternative (exploitation) and one or more unknown but potentially more rewarding alternatives (exploration). Prevailing theory on how humans perform the EE trade-off states that uncertainty is a major motivator for exploration: the more uncertain the environment, the more exploration that will occur. The current paper examines whether exploratory behaviour in both choice and attention may be impacted differently depending on whether uncertainty is onset suddenly (unexpected uncertainty), or more slowly (expected uncertainty). It is shown that when uncertainty was expected, participants tended to explore less with their choices, but not their attention, than when it was unexpected. Crucially, the impact of this "protection from uncertainty" on exploration only occurred when participants had an opportunity to learn the structure of the task prior to experiencing uncertainty. This suggests that the interaction between uncertainty and exploration is more nuanced than simply more uncertainty leading to more exploration, and that attention and choice behaviour may index separate aspects of the EE trade-off

Learned Predictiveness Models Predict Opposite Attention Biases in the Inverse Base-Rate Effect

Several attention-based models of associative learning are built upon the learned predictiveness principle, whereby learning is optimized by attending to the most predictive features and ignoring the least predictive features. Despite their functional similarity, these models differ in their formal mechanisms and thus may produce very different predictions in some circumstances. As we demonstrate, this is particularly evident in the inverse base-rate effect. Using simulations with a modified Mackintosh model and the EXIT model, we found that models based on the learned predictiveness principle can account for rare-outcome choice biases associated with the inverse base-rate effect, despite making opposite predictions for relative attention to rare versus common predictors. The models also make different predictions regarding changes in attention across training, and effects of context associations on attention to cues. Using a human causal learning task, we replicated the inverse base-rate effect and a recently reported reduction in this effect when the context is not predictive of the common outcome and used eye-tracking to test model predictions about changes in attention both prior to making a decision, and during feedback. The results support the predictions made by EXIT, where the rare predictor commands greater attention than the common predictor throughout training. In addition, patterns of attention prior to making a decision differed to those during feedback, where effects of using a partially predictive context were evident only prior to making a prediction

Testing the automaticity of an attentional bias towards predictive cues in human associative learning

It is well established that associative learning, such as learning new cue-outcome pairings, produces changes in attention: cues that are good predictors of relevant outcomes become prioritized compared to those that are non-predictive or redundant. However, there is controversy about whether such a learnt attentional bias results from a controlled orientation of attention, or whether it can be involuntary in nature. In three experiments, participants learned that cues of certain colours were predictive or non-predictive, and we assessed attention to cues using a dot-probe task. On dot-probe trials, participants were instructed to control attention by orienting towards a cue of a certain shape (target), while trying to ignore another cue (distractor). Although the colours of the cues were critical for the associative learning task, they were irrelevant for the dot-probe task. The results show that, even though participants’ controlled attention was focused on the target shape (as evident in response times and accuracy data), response times to the probe were slower (Experiments 1 and 2) and error rates were higher (Experiment 2 and 3) when the distractor was of a (previously) predictive colour. These data suggest that attention was captured involuntarily by the predictive value of the distractor, despite this being counterproductive to the task goal

Associative accounts of causal cognition

Humans are clearly sensitive to causal structures—we can describe and understand causal mechanisms and make predictions based on them. But this chapter asks: Is causal learning always causal? Or might seemingly causal behavior sometimes be based on associations that merely encode the information that two events “go together,” not that one causes the other? This associative view supposes that people often (mis)interpret associations as supporting the existence of a causal relationship between events; they make the everyday mistake of confusing correlation with causation. To assess the validity of this view, one must move away from considering specific implementations of associative models and instead focus on the general principle embodied by the associative approach—that the rules governing learning are general-purpose, and so do not differentiate between situations involving cause–effect relationships and those involving signaling relationships that are non-causal.</p

Failures of executive function when at a height:Negative height-related appraisals are associated with poor executive function during a virtual height stressor

It is difficult to maintain cognitive functioning in threatening contexts, even when it is imperative to do so. Research indicates that precarious situations can impair subsequent executive functioning, depending on whether they are appraised as threatening. Here, we used virtual reality to place participants at ground level or at a virtual height in order to examine the impact of a threat-related context on concurrent executive function and whether this relationship was modulated by negative appraisals of heights. Executive function was assessed via the Go/NoGo and N-Back tasks, indexing response inhibition and working memory updating respectively. Participants with negative appraisals of heights exhibited impaired executive function on both tasks when performing at a virtual height (i.e., a threat-related context) but not at ground-level, demonstrating the importance of considering the cognitive consequences of individual differences in negative interpretations of emotionally-evocative situations. We suggest that a virtual reality approach holds practical benefits for understanding how individuals are able to maintain cognitive ability when embedded within threatening situations

Contextual cuing of visual search does not guide attention automatically in the presence of top-down goals

Visual search is faster when it occurs within repeated displays, a phenomenon known as contextual cuing (CC). CC has been explained as the result of an automatic orientation of attention towards a target item driven by learned distractor-target associations. In three experiments we tested the specific hypothesis that CC is an automatic process of attentional guidance. Participants first searched for a T target in a standard CC procedure. Then, they experienced the same repeated configurations (with the T still present), but now searched for a Y target that was positioned either in a location on the same, or on a different side, from the old T target. Results suggested that there was no interference caused by the old T-target: target search was not affected by the relative positions of the T and Y. Instead, we found a general facilitation in search times for repeated configurations (Experiments 1 and 2). This main effect disappeared when the need for visual search was eliminated in Experiment 3 using a “feature search task”. These results suggest that repeated sets of distractors did not trigger an uncontrollable response towards the position of the T; instead, CC was produced by perceptual learning processes

Outcome predictability biases cued search

Within the domain of associative learning, there is substantial evidence that people (and other animals) select amongst environmental cues on the basis of their reinforcement history. Specifically, people preferentially attend to, and learn about, cueing stimuli that have previously predicted events of consequence (a predictiveness bias). By contrast, relatively little is known about whether people prioritize some (to-be-predicted) outcome events over others on the basis of their past experience with those outcomes (a predictability bias). The present experiments assessed whether the prior predictability of a stimulus results in a learning bias in a contingency learning task, as such effects are not anticipated by formal models of associative learning. Previously unpredictable stimuli were less readily learned about than previously predictable stimuli. This pattern is unlikely to reflect the use of strategic search processes or blocking of learning by the context. Instead we argue that our findings are most consistent with the operation of a biased learning mechanism

The role of uncertainty in attentional and choice exploration

The exploitation-exploration (EE) trade-off describes how, when making a decision, an organism must often choose between a safe alternative with a known pay-off, and one or more riskier alternatives with uncertain pay-offs. Recently, the concept of the EE trade-off has been extended to the examination of how organisms distribute limited attentional resources between several stimuli. This work suggests that when the rules governing the environment are certain, participants learn to “exploit” by attending preferentially to cues that provide the most information about upcoming events. However, when the rules are uncertain, people “explore” by increasing their attention to all cues that may provide information to help in predicting upcoming events. In the current study, we examine how uncertainty affects the EE trade-off in attention using a contextual two-armed bandit task, where participants explore with both their attention and their choice behavior. We find evidence for an influence of uncertainty on the EE trade-off in both choice and attention. These findings provide support for the idea of an EE trade-off in attention, and that uncertainty is a primary motivator for exploration in both choice and attentional allocation