Policy Improvement in Cribbage

Cribbage is a card game involving multiple methods of scoring which each receive varying emphasis over the course of a typical game. Reinforcement learning is a machine learning strategy in which an agent learns to accomplish a task via direct experience by collecting rewards based on performance. In this thesis, reinforcement learning is applied to the game of cribbage, improving an agent’s policy of combining multiple basic strategies, according to the needs of the dynamic state of the game. From inspection, a reasonable policy is learned by the agent over the course of a million games, but an increase in performance was not demonstrated

“Economic man” in cross-cultural perspective: Behavioral experiments in 15 small-scale societies

Researchers from across the social sciences have found consistent deviations from the predictions of the canonical model of self-interest in hundreds of experiments from around the world. This research, however, cannot determine whether the uniformity results from universal patterns of human behavior or from the limited cultural variation available among the university students used in virtually all prior experimental work. To address this, we undertook a cross-cultural study of behavior in ultimatum, public goods, and dictator games in a range of small-scale societies exhibiting a wide variety of economic and cultural conditions. We found, first, that the canonical model – based on self-interest – fails in all of the societies studied. Second, our data reveal substantially more behavioral variability across social groups than has been found in previous research. Third, group-level differences in economic organization and the structure of social interactions explain a substantial portion of the behavioral variation across societies: the higher the degree of market integration and the higher the payoffs to cooperation in everyday life, the greater the level of prosociality expressed in experimental games. Fourth, the available individual-level economic and demographic variables do not consistently explain game behavior, either within or across groups. Fifth, in many cases experimental play appears to reflect the common interactional patterns of everyday life

Towards Deep Learning with Competing Generalisation Objectives

The unreasonable effectiveness of Deep Learning continues to deliver unprecedented Artificial Intelligence capabilities to billions of people. Growing datasets and technological advances keep extending the reach of expressive model architectures trained through efficient optimisations. Thus, deep learning approaches continue to provide increasingly proficient subroutines for, among others, computer vision and natural interaction through speech and text. Due to their scalable learning and inference priors, higher performance is often gained cost-effectively through largely automatic training. As a result, new and improved capabilities empower more people while the costs of access drop. The arising opportunities and challenges have profoundly influenced research. Quality attributes of scalable software became central desiderata of deep learning paradigms, including reusability, efficiency, robustness and safety. Ongoing research into continual, meta- and robust learning aims to maximise such scalability metrics in addition to multiple generalisation criteria, despite possible conflicts. A significant challenge is to satisfy competing criteria automatically and cost-effectively. In this thesis, we introduce a unifying perspective on learning with competing generalisation objectives and make three additional contributions. When autonomous learning through multi-criteria optimisation is impractical, it is reasonable to ask whether knowledge of appropriate trade-offs could make it simultaneously effective and efficient. Informed by explicit trade-offs of interest to particular applications, we developed and evaluated bespoke model architecture priors. We introduced a novel architecture for sim-to-real transfer of robotic control policies by learning progressively to generalise anew. Competing desiderata of continual learning were balanced through disjoint capacity and hierarchical reuse of previously learnt representations. A new state-of-the-art meta-learning approach is then proposed. We showed that meta-trained hypernetworks efficiently store and flexibly reuse knowledge for new generalisation criteria through few-shot gradient-based optimisation. Finally, we characterised empirical trade-offs between the many desiderata of adversarial robustness and demonstrated a novel defensive capability of implicit neural networks to hinder many attacks simultaneously

Decisions, decisions, decisions: the development and plasticity of reinforcement learning, social and temporal decision making in children

Human decision-making is the flexible way people respond to their environment, take actions, and plan toward long-term goals. It is commonly thought that humans rely on distinct decision-making systems, which are either more habitual and reflexive or deliberate and calculated. How we make decisions can provide insight into our social functioning, mental health and underlying psychopathology, and ability to consider the consequences of our actions. Notably, the ability to make appropriate, habitual or deliberate decisions depending on the context, here referred to as metacontrol, remains underexplored in developmental samples. This thesis aims to investigate the development of different decision-making mechanisms in middle childhood (ages 5-13) and to illuminate the potential neurocognitive mechanisms underlying value-based decision-making. Using a novel sequential decision-making task, the first experimental chapter presents robust markers of model-based decision-making in childhood (N = 85), which reflects the ability to plan through a sequential task structure, contrary to previous developmental studies. Using the same paradigm, in a new sample via both behavioral (N = 69) and MRI-based measures (N = 44), the second experimental chapter explores the neurocognitive mechanisms that may underlie model-based decision-making and its metacontrol in childhood and links individual differences in inhibition and cortical thickness to metacontrol. The third experimental chapter explores the potential plasticity of social and intertemporal decision-making in a longitudinal executive function training paradigm (N = 205) and initial relationships with executive functions. Finally, I critically discuss the results presented in this thesis and their implications and outline directions for future research in the neurocognitive underpinnings of decision-making during development

Computational Theory of Mind for Human-Agent Coordination

In everyday life, people often depend on their theory of mind, i.e., their ability to reason about unobservable mental content of others to understand, explain, and predict their behaviour. Many agent-based models have been designed to develop computational theory of mind and analyze its effectiveness in various tasks and settings. However, most existing models are not generic (e.g., only applied in a given setting), not feasible (e.g., require too much information to be processed), or not human-inspired (e.g., do not capture the behavioral heuristics of humans). This hinders their applicability in many settings. Accordingly, we propose a new computational theory of mind, which captures the human decision heuristics of reasoning by abstracting individual beliefs about others. We specifically study computational affinity and show how it can be used in tandem with theory of mind reasoning when designing agent models for human-agent negotiation. We perform two-agent simulations to analyze the role of affinity in getting to agreements when there is a bound on the time to be spent for negotiating. Our results suggest that modeling affinity can ease the negotiation process by decreasing the number of rounds needed for an agreement as well as yield a higher benefit for agents with theory of mind reasoning.</p

Enhancing the implementation and sustainability of fundamental movement skill interventions in the UK and Ireland: lessons from collective intelligence engagement with stakeholders

Abstract Background To have population-level impact, physical activity (PA) interventions must be effectively implemented and sustained under real-world conditions. Adequate Fundamental Movement Skills (FMS) is integral to children being able to actively participate in play, games, and sports. Yet, few FMS interventions have been implemented at scale, nor sustained in routine practice, and thus it is important to understand the influences on sustained implementation. The study’s aim was to use Collective Intelligence (CI)—an applied systems science approach—with stakeholder groups to understand barriers to the implementation of FMS interventions, interdependencies between these barriers, and options to overcome the system of barriers identified. Methods Three CI sessions were conducted with three separate groups of experienced FMS intervention researchers/practitioners (N = 22) in the United Kingdom and Ireland. Participants generated and ranked barriers they perceive most critical in implementing FMS interventions. Each group developed a structural model describing how highly ranked barriers are interrelated in a system. Participants then conducted action mapping to solve the problem based on the logical relations between barriers reflected in the model. Results The top ranked barriers (of 76) are those related to policy, physical education curriculum, and stakeholders’ knowledge and appreciation. As reflected in the structural model, these barriers have influences over stakeholders’ efficacy in delivering and evaluating interventions. According to this logical structure, 38 solutions were created as a roadmap to inform policy, practice, and research. Collectively, solutions suggest that efforts in implementation and sustainability need to be coordinated (i.e., building interrelationship with multiple stakeholders), and a policy or local infrastructure that supports these efforts is needed. Conclusions The current study is the first to describe the complexity of barriers to implementing and sustaining FMS interventions and provide a roadmap of actions that help navigate through the complexity. By directing attention to the ecological context of FMS intervention research and participation, the study provides researchers, policy makers, and practitioners with a framework of critical components and players that need to be considered when designing and operationalising future projects in more systemic and relational terms