Changing the Environment Based on Empowerment as Intrinsic Motivation

This is an open access article distributed under the Creative Commons Attribution License CC BY 3.0 which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.One aspect of intelligence is the ability to restructure your own environment so that the world you live in becomes more beneficial to you. In this paper we investigate how the information-theoretic measure of agent empowerment can provide a task-independent, intrinsic motivation to restructure the world. We show how changes in embodiment and in the environment change the resulting behaviour of the agent and the artefacts left in the world. For this purpose, we introduce an approximation of the established empowerment formalism based on sparse sampling, which is simpler and significantly faster to compute for deterministic dynamics. Sparse sampling also introduces a degree of randomness into the decision making process, which turns out to beneficial for some cases. We then utilize the measure to generate agent behaviour for different agent embodiments in a Minecraft-inspired three dimensional block world. The paradigmatic results demonstrate that empowerment can be used as a suitable generic intrinsic motivation to not only generate actions in given static environments, as shown in the past, but also to modify existing environmental conditions. In doing so, the emerging strategies to modify an agent’s environment turn out to be meaningful to the specific agent capabilities, i.e., de facto to its embodiment.Peer reviewedFinal Published versio

Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks

Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself are crucial. In this work, we propose a novel framework for probabilistic online motion planning with online adaptation based on a bio-inspired stochastic recurrent neural network. By using learning signals which mimic the intrinsic motivation signalcognitive dissonance in addition with a mental replay strategy to intensify experiences, the stochastic recurrent network can learn from few physical interactions and adapts to novel environments in seconds. We evaluate our online planning and adaptation framework on an anthropomorphic KUKA LWR arm. The rapid online adaptation is shown by learning unknown workspace constraints sample-efficiently from few physical interactions while following given way points.Comment: accepted in Neural Network

Learning and Acting in Peripersonal Space: Moving, Reaching, and Grasping

The young infant explores its body, its sensorimotor system, and the immediately accessible parts of its environment, over the course of a few months creating a model of peripersonal space useful for reaching and grasping objects around it. Drawing on constraints from the empirical literature on infant behavior, we present a preliminary computational model of this learning process, implemented and evaluated on a physical robot. The learning agent explores the relationship between the configuration space of the arm, sensing joint angles through proprioception, and its visual perceptions of the hand and grippers. The resulting knowledge is represented as the peripersonal space (PPS) graph, where nodes represent states of the arm, edges represent safe movements, and paths represent safe trajectories from one pose to another. In our model, the learning process is driven by intrinsic motivation. When repeatedly performing an action, the agent learns the typical result, but also detects unusual outcomes, and is motivated to learn how to make those unusual results reliable. Arm motions typically leave the static background unchanged, but occasionally bump an object, changing its static position. The reach action is learned as a reliable way to bump and move an object in the environment. Similarly, once a reliable reach action is learned, it typically makes a quasi-static change in the environment, moving an object from one static position to another. The unusual outcome is that the object is accidentally grasped (thanks to the innate Palmar reflex), and thereafter moves dynamically with the hand. Learning to make grasps reliable is more complex than for reaches, but we demonstrate significant progress. Our current results are steps toward autonomous sensorimotor learning of motion, reaching, and grasping in peripersonal space, based on unguided exploration and intrinsic motivation.Comment: 35 pages, 13 figure

Competitive Engineering: Structural Climate Modifications To Enhance Youth Athletes\u27 Competitive Experience

Competitive engineering (CE) is a structural-based approach to changing the competitive environment of youth sports to provide more nurturing competitive experiences. Thus, in youth sport, CE attempts to enhance a variety of psychosocial outcomes by making systematic changes to the competitive environment in which athletes perform. A working CE model is presented that employs four CE strategies (i.e., modifying structure, rules, facilities and equipment) to promote athlete engagement goals based on athlete-directed sandlot sport principles (i.e., increasing action and scoring, keeping scores close, enhancing personal involvement, and maintaining positive social relationships) in order to attain intrinsic motivation outcomes, particularly competence, autonomy, relatedness and Row while promoting an autonomy supportive climate. Discussion focuses on how the CE model can best promote research and intervention to enhance competitive climates in order to promote better sport experiences for all youngsters

Knowledge Creation and Sharing in Organisational Contexts: A Motivation-Based Perspective

This paper develops a motivation-based perspective to explore how organisations resolve the social dilemma of knowledge sharing, and the ways in which different motivational mechanisms interact to foster knowledge sharing and creation in different organisational contexts. The core assumption is that the willingness of organisational members to engage in knowledge sharing can be viewed on a continuum from purely opportunistic behaviour regulated by extrinsic incentives to an apparently altruistic stance fostered by social norms and group identity. The analysis builds on a three-category taxonomy of motivation: adding ‘hedonic’ motivation to the traditional dichotomy of extrinsic and intrinsic motivation. Based on an analysis of empirical case studies in the literature, we argue that the interaction and mix of the three different motivators play a key role in regulating and translating potential into actual behaviour, and they underline the complex dynamics of knowledge sharing and creation in different organisational contexts

Making things happen : a model of proactive motivation

Being proactive is about making things happen, anticipating and preventing problems, and seizing opportunities. It involves self-initiated efforts to bring about change in the work environment and/or oneself to achieve a different future. The authors develop existing perspectives on this topic by identifying proactivity as a goal-driven process involving both the setting of a proactive goal (proactive goal generation) and striving to achieve that proactive goal (proactive goal striving). The authors identify a range of proactive goals that individuals can pursue in organizations. These vary on two dimensions: the future they aim to bring about (achieving a better personal fit within one’s work environment, improving the organization’s internal functioning, or enhancing the organization’s strategic fit with its environment) and whether the self or situation is being changed. The authors then identify “can do,” “reason to,” and “energized to” motivational states that prompt proactive goal generation and sustain goal striving. Can do motivation arises from perceptions of self-efficacy, control, and (low) cost. Reason to motivation relates to why someone is proactive, including reasons flowing from intrinsic, integrated, and identified motivation. Energized to motivation refers to activated positive affective states that prompt proactive goal processes. The authors suggest more distal antecedents, including individual differences (e.g., personality, values, knowledge and ability) as well as contextual variations in leadership, work design, and interpersonal climate, that influence the proactive motivational states and thereby boost or inhibit proactive goal processes. Finally, the authors summarize priorities for future researc

Motivating the construction academic: a conceptual study

The main purpose of this study is to understand factors that motivate and demotivate a construction academic based on existing literature. An extensive examination of published literature failed to reveal any studies on motivation or demotivation of construction academics but for a few studies on motivation of academics in general. These studies revealed over 25 intrinsic and extrinsic factors which were differentiated between factors cited in conceptual and empirical studies. A further distinction was made between factors cited in studies focussed directly on motivation of academics, and factors cited in studies investigating a different topic. Factors so identified, provide a broad base for understanding ‘what’ factors affect motivation and demotivation of academics However, these studies have not taken into account discipline specific, job level, and other contextual issues or prioritised factors based on importance. Moreover, ‘how’ these factors could be used for improving organisational performance focussing on different disciplines and roles within these disciplines have not been studied either. Nevertheless, an examination of these factors revealed that most fall within the control of the university management. As such, there is a need for understanding what management styles could be used for increasing motivation and minimising demotivation, and this is an area that needs investigation focussing on construction specific issues vis-à-vis context and job roles

Combining Experience Replay with Exploration by Random Network Distillation

Our work is a simple extension of the paper "Exploration by Random Network Distillation". More in detail, we show how to efficiently combine Intrinsic Rewards with Experience Replay in order to achieve more efficient and robust exploration (with respect to PPO/RND) and consequently better results in terms of agent performances and sample efficiency. We are able to do it by using a new technique named Prioritized Oversampled Experience Replay (POER), that has been built upon the definition of what is the important experience useful to replay. Finally, we evaluate our technique on the famous Atari game Montezuma's Revenge and some other hard exploration Atari games.Comment: 8 pages, 6 figures, accepted as full-paper at IEEE Conference on Games (CoG) 201