30,153 research outputs found
Perceptual Attention-based Predictive Control
In this paper, we present a novel information processing architecture for
safe deep learning-based visual navigation of autonomous systems. The proposed
information processing architecture is used to support a perceptual
attention-based predictive control algorithm that leverages model predictive
control (MPC), convolutional neural networks (CNNs), and uncertainty
quantification methods. The novelty of our approach lies in using MPC to learn
how to place attention on relevant areas of the visual input, which ultimately
allows the system to more rapidly detect unsafe conditions. We accomplish this
by using MPC to learn to select regions of interest in the input image, which
are used to output control actions as well as estimates of epistemic and
aleatoric uncertainty in the attention-aware visual input. We use these
uncertainty estimates to quantify the safety of our network controller under
the current navigation condition. The proposed architecture and algorithm is
tested on a 1:5 scale terrestrial vehicle. Experimental results show that the
proposed algorithm outperforms previous approaches on early detection of unsafe
conditions, such as when novel obstacles are present in the navigation
environment. The proposed architecture is the first step towards using deep
learning-based perceptual control policies in safety-critical domains
Emotion in Reinforcement Learning Agents and Robots: A Survey
This article provides the first survey of computational models of emotion in
reinforcement learning (RL) agents. The survey focuses on agent/robot emotions,
and mostly ignores human user emotions. Emotions are recognized as functional
in decision-making by influencing motivation and action selection. Therefore,
computational emotion models are usually grounded in the agent's decision
making architecture, of which RL is an important subclass. Studying emotions in
RL-based agents is useful for three research fields. For machine learning (ML)
researchers, emotion models may improve learning efficiency. For the
interactive ML and human-robot interaction (HRI) community, emotions can
communicate state and enhance user investment. Lastly, it allows affective
modelling (AM) researchers to investigate their emotion theories in a
successful AI agent class. This survey provides background on emotion theory
and RL. It systematically addresses 1) from what underlying dimensions (e.g.,
homeostasis, appraisal) emotions can be derived and how these can be modelled
in RL-agents, 2) what types of emotions have been derived from these
dimensions, and 3) how these emotions may either influence the learning
efficiency of the agent or be useful as social signals. We also systematically
compare evaluation criteria, and draw connections to important RL sub-domains
like (intrinsic) motivation and model-based RL. In short, this survey provides
both a practical overview for engineers wanting to implement emotions in their
RL agents, and identifies challenges and directions for future emotion-RL
research.Comment: To be published in Machine Learning Journa
Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication
Deep reinforcement learning algorithms have recently been used to train
multiple interacting agents in a centralised manner whilst keeping their
execution decentralised. When the agents can only acquire partial observations
and are faced with tasks requiring coordination and synchronisation skills,
inter-agent communication plays an essential role. In this work, we propose a
framework for multi-agent training using deep deterministic policy gradients
that enables concurrent, end-to-end learning of an explicit communication
protocol through a memory device. During training, the agents learn to perform
read and write operations enabling them to infer a shared representation of the
world. We empirically demonstrate that concurrent learning of the communication
device and individual policies can improve inter-agent coordination and
performance in small-scale systems. Our experimental results show that the
proposed method achieves superior performance in scenarios with up to six
agents. We illustrate how different communication patterns can emerge on six
different tasks of increasing complexity. Furthermore, we study the effects of
corrupting the communication channel, provide a visualisation of the
time-varying memory content as the underlying task is being solved and validate
the building blocks of the proposed memory device through ablation studies
Evolving Indoor Navigational Strategies Using Gated Recurrent Units In NEAT
Simultaneous Localisation and Mapping (SLAM) algorithms are expensive to run
on smaller robotic platforms such as Micro-Aerial Vehicles. Bug algorithms are
an alternative that use relatively little processing power, and avoid high
memory consumption by not building an explicit map of the environment. Bug
Algorithms achieve relatively good performance in simulated and robotic maze
solving domains. However, because they are hand-designed, a natural question is
whether they are globally optimal control policies. In this work we explore the
performance of Neuroevolution - specifically NEAT - at evolving control
policies for simulated differential drive robots carrying out generalised maze
navigation. We extend NEAT to include Gated Recurrent Units (GRUs) to help deal
with long term dependencies. We show that both NEAT and our NEAT-GRU can
repeatably generate controllers that outperform I-Bug (an algorithm
particularly well-suited for use in real robots) on a test set of 209 indoor
maze like environments. We show that NEAT-GRU is superior to NEAT in this task
but also that out of the 2 systems, only NEAT-GRU can continuously evolve
successful controllers for a much harder task in which no bearing information
about the target is provided to the agent
When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
With widespread applications of artificial intelligence (AI), the
capabilities of the perception, understanding, decision-making and control for
autonomous systems have improved significantly in the past years. When
autonomous systems consider the performance of accuracy and transferability,
several AI methods, like adversarial learning, reinforcement learning (RL) and
meta-learning, show their powerful performance. Here, we review the
learning-based approaches in autonomous systems from the perspectives of
accuracy and transferability. Accuracy means that a well-trained model shows
good results during the testing phase, in which the testing set shares a same
task or a data distribution with the training set. Transferability means that
when a well-trained model is transferred to other testing domains, the accuracy
is still good. Firstly, we introduce some basic concepts of transfer learning
and then present some preliminaries of adversarial learning, RL and
meta-learning. Secondly, we focus on reviewing the accuracy or transferability
or both of them to show the advantages of adversarial learning, like generative
adversarial networks (GANs), in typical computer vision tasks in autonomous
systems, including image style transfer, image superresolution, image
deblurring/dehazing/rain removal, semantic segmentation, depth estimation,
pedestrian detection and person re-identification (re-ID). Then, we further
review the performance of RL and meta-learning from the aspects of accuracy or
transferability or both of them in autonomous systems, involving pedestrian
tracking, robot navigation and robotic manipulation. Finally, we discuss
several challenges and future topics for using adversarial learning, RL and
meta-learning in autonomous systems
A Comprehensive Review of Shepherding as a Bio-inspired Swarm-Robotics Guidance Approach
The simultaneous control of multiple coordinated robotic agents represents an
elaborate problem. If solved, however, the interaction between the agents can
lead to solutions to sophisticated problems. The concept of swarming, inspired
by nature, can be described as the emergence of complex system-level behaviors
from the interactions of relatively elementary agents. Due to the effectiveness
of solutions found in nature, bio-inspired swarming-based control techniques
are receiving a lot of attention in robotics. One method, known as swarm
shepherding, is founded on the sheep herding behavior exhibited by sheepdogs,
where a swarm of relatively simple agents are governed by a shepherd (or
shepherds) which is responsible for high-level guidance and planning. Many
studies have been conducted on shepherding as a control technique, ranging from
the replication of sheep herding via simulation, to the control of uninhabited
vehicles and robots for a variety of applications. We present a comprehensive
review of the literature on swarm shepherding to reveal the advantages and
potential of the approach to be applied to a plethora of robotic systems in the
future.Comment: Copyright 2020 IEEE. Personal use of this material is permitted.
Permission from IEEE must be obtained for all other uses, in any current or
future media, including reprinting/republishing this material for advertising
or promotional purposes, creating new collective works, for resale or
redistribution to servers or lists, or reuse of any copyrighted component of
this work in other work
A Brief Survey of Deep Reinforcement Learning
Deep reinforcement learning is poised to revolutionise the field of AI and
represents a step towards building autonomous systems with a higher level
understanding of the visual world. Currently, deep learning is enabling
reinforcement learning to scale to problems that were previously intractable,
such as learning to play video games directly from pixels. Deep reinforcement
learning algorithms are also applied to robotics, allowing control policies for
robots to be learned directly from camera inputs in the real world. In this
survey, we begin with an introduction to the general field of reinforcement
learning, then progress to the main streams of value-based and policy-based
methods. Our survey will cover central algorithms in deep reinforcement
learning, including the deep -network, trust region policy optimisation, and
asynchronous advantage actor-critic. In parallel, we highlight the unique
advantages of deep neural networks, focusing on visual understanding via
reinforcement learning. To conclude, we describe several current areas of
research within the field.Comment: IEEE Signal Processing Magazine, Special Issue on Deep Learning for
Image Understanding (arXiv extended version
A Survey of Deep Learning Techniques for Mobile Robot Applications
Advancements in deep learning over the years have attracted research into how
deep artificial neural networks can be used in robotic systems. This research
survey will present a summarization of the current research with a specific
focus on the gains and obstacles for deep learning to be applied to mobile
robotics
Exploring applications of deep reinforcement learning for real-world autonomous driving systems
Deep Reinforcement Learning (DRL) has become increasingly powerful in recent
years, with notable achievements such as Deepmind's AlphaGo. It has been
successfully deployed in commercial vehicles like Mobileye's path planning
system. However, a vast majority of work on DRL is focused on toy examples in
controlled synthetic car simulator environments such as TORCS and CARLA. In
general, DRL is still at its infancy in terms of usability in real-world
applications. Our goal in this paper is to encourage real-world deployment of
DRL in various autonomous driving (AD) applications. We first provide an
overview of the tasks in autonomous driving systems, reinforcement learning
algorithms and applications of DRL to AD systems. We then discuss the
challenges which must be addressed to enable further progress towards
real-world deployment.Comment: Accepted for Oral Presentation at VISAPP 201
Using Digital Twins and Intelligent Cognitive Agencies to Build Platforms for Automated CxO Future of Work
AI, Algorithms and Machine based automation of executive functions in
enterprises and institutions is an important niche in the current
considerations about the impact of digitalization on the future of work.
Building platforms for CxO automation is challenging. In this paper, design
principles based on computational thinking are used to engineer the
architecture and infrastructure for such CxO automation platforms
- …