Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds

We describe a method to use discrete human feedback to enhance the performance of deep learning agents in virtual three-dimensional environments by extending deep-reinforcement learning to model the confidence and consistency of human feedback. This enables deep reinforcement learning algorithms to determine the most appropriate time to listen to the human feedback, exploit the current policy model, or explore the agent's environment. Managing the trade-off between these three strategies allows DRL agents to be robust to inconsistent or intermittent human feedback. Through experimentation using a synthetic oracle, we show that our technique improves the training speed and overall performance of deep reinforcement learning in navigating three-dimensional environments using Minecraft. We further show that our technique is robust to highly innacurate human feedback and can also operate when no human feedback is given

The NetHack learning environment

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both. Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open source and available at https://github.com/facebookresearch/nle

Literacy for digital futures : Mind, body, text

The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualised, embodied, multimodal, and digitally mediated. In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes – Mind and Materiality; Body and Senses; and Texts and Digital Semiotics – to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice

AI in Learning: Designing the Future

AI (Artificial Intelligence) is predicted to radically change teaching and learning in both schools and industry causing radical disruption of work. AI can support well-being initiatives and lifelong learning but educational institutions and companies need to take the changing technology into account. Moving towards AI supported by digital tools requires a dramatic shift in the concept of learning, expertise and the businesses built off of it. Based on the latest research on AI and how it is changing learning and education, this book will focus on the enormous opportunities to expand educational settings with AI for learning in and beyond the traditional classroom. This open access book also introduces ethical challenges related to learning and education, while connecting human learning and machine learning. This book will be of use to a variety of readers, including researchers, AI users, companies and policy makers

AI in Learning: Designing the Future

AI (Artificial Intelligence) is predicted to radically change teaching and learning in both schools and industry causing radical disruption of work. AI can support well-being initiatives and lifelong learning but educational institutions and companies need to take the changing technology into account. Moving towards AI supported by digital tools requires a dramatic shift in the concept of learning, expertise and the businesses built off of it. Based on the latest research on AI and how it is changing learning and education, this book will focus on the enormous opportunities to expand educational settings with AI for learning in and beyond the traditional classroom. This open access book also introduces ethical challenges related to learning and education, while connecting human learning and machine learning. This book will be of use to a variety of readers, including researchers, AI users, companies and policy makers

A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks

Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync .Comment: Accepted to ECCV 2020 (spotlight); Project page: https://unnat.github.io/cordial-syn

Envisioning the future village: the role of digital technology in supporting more inclusive visions in the neighbourhood planning process

This thesis presents the development of a digitally aided Collaborative Envisioning Framework, to support disenfranchised young people in contributing to a ‘shared vision’ of their community’s future. Drawing from the research areas of planning, design, collaboration and envisioning, this study sought to address the existing democratic deficit in local decision making activities, by utilising the new potentials of digital technologies. The research aim was to support communities, particularly disengaged young people, in becoming involved with decision-making activities, namely generating a shared vision for a neighbourhood plan. Since the radical policy changes to the National Planning Policy Framework and Localism Act 2011, members of the public have been handed increased responsibility and accountability in contributing to the local decisions affecting them. However, the tools and resources have been criticized for not engaging and including all sectors of the public, particularly young people (who arguably have the most to gain, or lose, as a result of decisions made). Using community and neighbourhood planning as a microcosm of a larger problem, this study looked towards the potentials of digital tools as a way to address this democratic deficit. To discover whether they offered anything more than existing tools, by helping young people to contribute to the generation of a ‘shared vision’ (a requisite of a neighbourhood planning application). It also addressed the assumption that the public had an understanding of what creating a ‘shared vision’ entailed, and had the skills and knowledge required to create one. It firstly identified envisioning as a design activity, which needs creativity, imagination, empathy, collaboration, communication and deliberation, and then identified ‘designable factors’ such as processes, tools (digital and non-digital), environments, and services which are able to support these, focusing on which were most suitable for the young audience. The research also explored behavior and motivation theories which guided the design of an envisioning framework. To achieve this aim, a constructive design research methodology was adopted consisting of a designed artefact - ‘The Collaborative Envisioning Framework’ which was utlised throughout numerous workshops. The interactions between the workshop participants and the envisioning framework generated multiple sets of qualitative data, which were analysed and interpreted to form the next iteration of the framework. The research demonstrates that existing tools and resources aimed at supporting inclusivity and meaningful visions for neighbourhood plans are not, in their current form, adequate to firstly, engage the diverse groups of people they should be including, and secondly, to support a generative, creative activity of envisioning, and suggests that the use of digital tools (namely Ageing Booth App, Morfo App, and Minecraft) offer something new. The original contributions to knowledge are: an advancement of constructive design research methodology; contributions to the discourse surrounding the purpose and value of visons within community planning; and a practical ‘Collaborative Envisioning Framework’ which can be followed by public sector and private organisations who seek to support communities in producing ‘visons’ for their community

Computer Game Innovation

Faculty of Technical Physics, Information Technology and Applied Mathematics. Institute of Information TechnologyWydział Fizyki Technicznej, Informatyki i Matematyki Stosowanej. Instytut InformatykiThe "Computer Game Innovations" series is an international forum designed to enable the exchange of knowledge and expertise in the field of video game development. Comprising both academic research and industrial needs, the series aims at advancing innovative industry-academia collaboration. The monograph provides a unique set of articles presenting original research conducted in the leading academic centres which specialise in video games education. The goal of the publication is, among others, to enhance networking opportunities for industry and university representatives seeking to form R&D partnerships. This publication covers the key focus areas specified in the GAMEINN sectoral programme supported by the National Centre for Research and Development