Learning Contextual Reward Expectations for Value Adaptation

Substantial evidence indicates that subjective value is adapted to the statistics of reward expected within a given temporal context. However, how these contextual expectations are learned is poorly understood. To examine such learning, we exploited a recent observation that participants performing a gambling task adjust their preferences as a function of context. We show that, in the absence of contextual cues providing reward information, an average reward expectation was learned from recent past experience. Learning dependent on contextual cues emerged when two contexts alternated at a fast rate, whereas both cue-independent and cue-dependent forms of learning were apparent when two contexts alternated at a slower rate. Motivated by these behavioral findings, we reanalyzed a previous fMRI data set to probe the neural substrates of learning contextual reward expectations. We observed a form of reward prediction error related to average reward such that, at option presentation, activity in ventral tegmental area/substantia nigra and ventral striatum correlated positively and negatively, respectively, with the actual and predicted value of options. Moreover, an inverse correlation between activity in ventral tegmental area/substantia nigra (but not striatum) and predicted option value was greater in participants showing enhanced choice adaptation to context. The findings help understanding the mechanisms underlying learning of contextual reward expectation

Self-directedness, integration and higher cognition

In this paper I discuss connections between self-directedness, integration and higher cognition. I present a model of self-directedness as a basis for approaching higher cognition from a situated cognition perspective. According to this model increases in sensorimotor complexity create pressure for integrative higher order control and learning processes for acquiring information about the context in which action occurs. This generates complex articulated abstractive information processing, which forms the major basis for higher cognition. I present evidence that indicates that the same integrative characteristics found in lower cognitive process such as motor adaptation are present in a range of higher cognitive process, including conceptual learning. This account helps explain situated cognition phenomena in humans because the integrative processes by which the brain adapts to control interaction are relatively agnostic concerning the source of the structure participating in the process. Thus, from the perspective of the motor control system using a tool is not fundamentally different to simply controlling an arm

How the visual environment shapes attention: The role of context in attention guidance

In our environment, visual stimuli typically appear within the context of other stimuli, which are usually not arranged randomly but follow regularities. These regularities can be very useful for the visual system to overcome the problem of limited encoding capacity by guiding attention to stimuli which are relevant for behavior. There is growing evidence that observers use repeated contexts for guiding attention in visual search, and there is evidence that observers adapt to dynamical changes in their visual environment. However, contexts in our natural environment often come with features predicting reward, and little is known about the influence of such reward-predicting contexts on attention guidance. In addition, it is unclear how observers adapt their behavior to context features that are not relevant for the task, and little is known about individual differences in the effects of contexts. These research gaps are addressed in the present dissertation. In five studies, the present dissertation investigates how different types of contextual regularities are integrated into behavior and how these regularities guide visual attention. Study I showed that observers use knowledge they have acquired in former encounters with similar scenes to predict the most promising item to attend to in an upcoming scene. In a visual search task in the laboratory, participants responded faster in visual contexts that repeated compared to contexts that were novel. In addition, they also moved their eyes more efficiently to the target when they encountered repeated contexts. These results suggest that participants use repeated visual contexts to learn to predict the target location. Study I also revealed that visual contexts are especially used for specifying promising items when they predict a high reward. Context features predicting a high reward boosted the performance advantages observed with repeated contexts. This result suggests that the prediction of reward facilitates the generation of expectations about potential target locations. Study II demonstrated that expectations about potential target locations were quite persistent, since performance benefits were observed even after many encounters with repeated contexts. Further experiments showed that participants could use even a very limited part of the visual contexts to learn to predict the target location (Study III) and that observers used also contexts that changed dynamically for specifying promising items to attend to (Study IV). These results suggest that observers use regularities in the visual context to generate expectations about promising items in their visual environment. Finally, the last study of this dissertation (Study V) investigated how contexts of social perception are used for specifying relevant visual information. Results showed that observers differ in how they use contexts for specifying relevant visual information and suggested that an observer’s personality might be one factor explaining these differences. In sum, the five studies of the present dissertation demonstrate that the visual system is remarkably sensitive to regularities in the visual context. It is quite efficient in extracting repeated contexts to guide attention to relevant locations when contexts are encountered again (Studies I and II), and it only needs a very limited amount of repeating contextual information to take advantage from the contexts (Study III). It also considers rewards that are signaled by features of the contexts to prioritize processing of high reward contexts. The visual system further adapts to dynamical changes in the contexts (Study IV) and uses contexts of social perception for prioritizing information, dependent on the observer’s personality (Study V). The present dissertation thus highlights that the visual context is crucial for guiding our attention in numerous situations that we encounter every day. Fortunately, we can take advantage of the visual context, which allows our visual system to cope with its limited processing capacity

A Bayesian model of context-sensitive value attribution

Substantial evidence indicates that incentive value depends on an anticipation of rewards within a given context. However, the computations underlying this context sensitivity remain unknown. To address this question, we introduce a normative (Bayesian) account of how rewards map to incentive values. This assumes that the brain inverts a model of how rewards are generated. Key features of our account include (i) an influence of prior beliefs about the context in which rewards are delivered (weighted by their reliability in a Bayes-optimal fashion), (ii) the notion that incentive values correspond to precision-weighted prediction errors, (iii) and contextual information unfolding at different hierarchical levels. This formulation implies that incentive value is intrinsically context-dependent. We provide empirical support for this model by showing that incentive value is influenced by context variability and by hierarchically nested contexts. The perspective we introduce generates new empirical predictions that might help explaining psychopathologies, such as addiction

"We are always after that balance":managing innovation in the new digital media industry

The pressure to innovate is growing as technology cycles change more rapidly. Organisations need to balance exploration and exploitation effectively if they are to heed the innovation imperative. Organisational ambidexterity is proposed as a means to achieve such balance with structural or contextual ambidexterity as possible choices. Yet how organisations become ambidextrous is an as yet underresearched area, and different industry sectors may pose different innovation challenges. Using the case study method, this paper examines how a computer games company responds to an industry-specific innovation challenge and how it endeavours to balance exploration and exploitation. The findings suggest that ambidexterity is difficult to achieve, and is fraught with organisational tensions which might eventually jeopardise the innovation potential of a company. The paper suggests that more qualitative research is needed to further our understanding of innovation challenges, innovation management and organisational ambidexterity