Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients

This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We offer an instrumental view of action selection, in which an action serves to disclose outcomes that have intrinsic meaningfulness to an agent itself. This motivated two central algorithmic ingredients: devaluation and devaluation progress, both underpin agent's cognition concerning intrinsically generated rewards. The two serve as an instantiation of homeostatic and heterostatic intrinsic motivation. A key insight from our algorithm is that the two seemingly opposite motivations can be reconciled—without which exploration and information-gathering cannot be effectively carried out. We supported this claim with empirical evidence, showing that boredom-enabled agents consistently outperformed other curious or explorative agent variants in model building benchmarks based on self-assisted experience accumulation

The Desire to Act: Exploring Situational, Dispositional and Genetic Correlates of a Fundamental Motivational State

Animals do not simply act to survive and maximize pleasure. They also act for the sake of action itself. Although such intrinsically motivated actions are ubiquitous throughout the animal kingdom, the mechanisms by which they are enacted remain poorly understood. Likewise, little is known in regard to what influences which actions an animal ultimately chooses. It has been speculated for some time that boredom signals our failure to satisfy this drive to act, and that it may play a crucial role in launching us into action. The goal of the current thesis was to investigate the role of boredom in mediating our desire to engage with our environment and to explore factors that influence how we interact with our environment. Chapter 2 tested whether boredom signals opportunities for action and makes us want to engage with such opportunities. Chapter 3 investigated whether different situations make us sensitive to distinct opportunities to act. Chapter 4 investigated whether a genetic variation predisposes us to be sensitive to specific opportunities to act. The results from this thesis highlight the importance of boredom in regulating our desire for action and provide novel insights into what factors make us act in specific ways. The last chapter situates these results within a broader self-regulatory framework