Influencing Exploration in Actor-Critic Reinforcement Learning Algorithms

Reinforcement Learning (RL) is a subset of machine learning primarily concerned with goal-directed learning and optimal decision making. RL agents learn based on a reward signal discovered from trial and error in complex, uncertain environments with the goal of maximizing positive reward signals. RL approaches need to scale up as they are applied to more complex environments with extremely large state spaces. Inefficient exploration methods cannot sufficiently explore complex environments in a reasonable amount of time, and optimal policies will be unrealized resulting in RL agents failing to solve an environment. This thesis proposes a novel variant of the Actor-Advantage Critic (A2C) algorithm. The variant is validated against two state-of-the-art RL algorithms, Deep Q-Network (DQN) and A2C, across six Atari 2600 games of varying difficulty. The experimental results are competitive with state-of-the-art and achieve lower variance and quicker learning speed. Additionally, the thesis introduces a metric to objectively quantify the difficulty of any Markovian environment with respect to the exploratory capacity of RL agents

Makers at School, Educational Robotics and Innovative Learning Environments

This open access book contains observations, outlines, and analyses of educational robotics methodologies and activities, and developments in the field of educational robotics emerging from the findings presented at FabLearn Italy 2019, the international conference that brought together researchers, teachers, educators and practitioners to discuss the principles of Making and educational robotics in formal, non-formal and informal education. The editors’ analysis of these extended versions of papers presented at FabLearn Italy 2019 highlight the latest findings on learning models based on Making and educational robotics. The authors investigate how innovative educational tools and methodologies can support a novel, more effective and more inclusive learner-centered approach to education. The following key topics are the focus of discussion: Makerspaces and Fab Labs in schools, a maker approach to teaching and learning; laboratory teaching and the maker approach, models, methods and instruments; curricular and non-curricular robotics in formal, non-formal and informal education; social and assistive robotics in education; the effect of innovative spaces and learning environments on the innovation of teaching, good practices and pilot projects

Multi Agent Systems

Research on multi-agent systems is enlarging our future technical capabilities as humans and as an intelligent society. During recent years many effective applications have been implemented and are part of our daily life. These applications have agent-based models and methods as an important ingredient. Markets, finance world, robotics, medical technology, social negotiation, video games, big-data science, etc. are some of the branches where the knowledge gained through multi-agent simulations is necessary and where new software engineering tools are continuously created and tested in order to reach an effective technology transfer to impact our lives. This book brings together researchers working in several fields that cover the techniques, the challenges and the applications of multi-agent systems in a wide variety of aspects related to learning algorithms for different devices such as vehicles, robots and drones, computational optimization to reach a more efficient energy distribution in power grids and the use of social networks and decision strategies applied to the smart learning and education environments in emergent countries. We hope that this book can be useful and become a guide or reference to an audience interested in the developments and applications of multi-agent systems

Makers at School, Educational Robotics and Innovative Learning Environments

This open access book contains observations, outlines, and analyses of educational robotics methodologies and activities, and developments in the field of educational robotics emerging from the findings presented at FabLearn Italy 2019, the international conference that brought together researchers, teachers, educators and practitioners to discuss the principles of Making and educational robotics in formal, non-formal and informal education. The editors’ analysis of these extended versions of papers presented at FabLearn Italy 2019 highlight the latest findings on learning models based on Making and educational robotics. The authors investigate how innovative educational tools and methodologies can support a novel, more effective and more inclusive learner-centered approach to education. The following key topics are the focus of discussion: Makerspaces and Fab Labs in schools, a maker approach to teaching and learning; laboratory teaching and the maker approach, models, methods and instruments; curricular and non-curricular robotics in formal, non-formal and informal education; social and assistive robotics in education; the effect of innovative spaces and learning environments on the innovation of teaching, good practices and pilot projects

Diogene-CT: tools and methodologies for teaching and learning coding

AbstractComputational thinking is the capacity of undertaking a problem-solving process in various disciplines (including STEM, i.e. science, technology, engineering and mathematics) using distinctive techniques that are typical of computer science. It is nowadays considered a fundamental skill for students and citizens, that has the potential to affect future generations. At the roots of computational-thinking abilities stands the knowledge of computer programming, i.e. coding. With the goal of fostering computational thinking in young students, we address the challenging and open problem of using methods, tools and techniques to support teaching and learning of computer-programming skills in school curricula of the secondary grade and university courses. This problem is made complex by several factors. In fact, coding requires abstraction capabilities and complex cognitive skills such as procedural and conditional reasoning, planning, and analogical reasoning. In this paper, we introduce a new paradigm called ACME ("Code Animation by Evolved Metaphors") that stands at the foundation of the Diogene-CT code visualization environment and methodology. We develop consistent visual metaphors for both procedural and object-oriented programming. Based on the metaphors, we introduce a playground architecture to support teaching and learning of the principles of coding. To the best of our knowledge, this is the first scalable code visualization tool using consistent metaphors in the field of the Computing Education Research (CER). It might be considered as a new kind of tools named as code visualization environments

Diogene-CT: tools and methodologies for teaching and learning coding

Computational thinking is the capacity of undertaking a problem-solving process in various disciplines (including STEM, i.e. science, technology, engineering and mathematics) using distinctive techniques that are typical of computer science. It is nowadays considered a fundamental skill for students and citizens, that has the potential to affect future generations. At the roots of computational-thinking abilities stands the knowledge of computer programming, i.e. coding. With the goal of fostering computational thinking in young students, we address the challenging and open problem of using methods, tools and techniques to support teaching and learning of computer-programming skills in school curricula of the secondary grade and university courses. This problem is made complex by several factors. In fact, coding requires abstraction capabilities and complex cognitive skills such as procedural and conditional reasoning, planning, and analogical reasoning. In this paper, we introduce a new paradigm called ACME (“Code Animation by Evolved Metaphors”) that stands at the foundation of the Diogene-CT code visualization environment and methodology. We develop consistent visual metaphors for both procedural and object-oriented programming. Based on the metaphors, we introduce a playground architecture to support teaching and learning of the principles of coding. To the best of our knowledge, this is the first scalable code visualization tool using consistent metaphors in the field of the Computing Education Research (CER). It might be considered as a new kind of tools named as code visualization environments