725 research outputs found
Explainability in Deep Reinforcement Learning
A large set of the explainable Artificial Intelligence (XAI) literature is
emerging on feature relevance techniques to explain a deep neural network (DNN)
output or explaining models that ingest image source data. However, assessing
how XAI techniques can help understand models beyond classification tasks, e.g.
for reinforcement learning (RL), has not been extensively studied. We review
recent works in the direction to attain Explainable Reinforcement Learning
(XRL), a relatively new subfield of Explainable Artificial Intelligence,
intended to be used in general public applications, with diverse audiences,
requiring ethical, responsible and trustable algorithms. In critical situations
where it is essential to justify and explain the agent's behaviour, better
explainability and interpretability of RL models could help gain scientific
insight on the inner workings of what is still considered a black box. We
evaluate mainly studies directly linking explainability to RL, and split these
into two categories according to the way the explanations are generated:
transparent algorithms and post-hoc explainaility. We also review the most
prominent XAI works from the lenses of how they could potentially enlighten the
further deployment of the latest advances in RL, in the demanding present and
future of everyday problems.Comment: Article accepted at Knowledge-Based System
Machine learning for optical fiber communication systems: An introduction and overview
Optical networks generate a vast amount of diagnostic, control and performance monitoring data. When information is
extracted from this data, reconfigurable network elements and reconfigurable transceivers allow the network to adapt
both to changes in the physical infrastructure but also changing traffic conditions. Machine learning is emerging as a
disruptive technology for extracting useful information from this raw data to enable enhanced planning, monitoring and
dynamic control. We provide a survey of the recent literature and highlight numerous promising avenues for machine
learning applied to optical networks, including explainable machine learning, digital twins and approaches in which we
embed our knowledge into the machine learning such as physics-informed machine learning for the physical layer and
graph-based machine learning for the networking layer
On Reducing Undesirable Behavior in Deep Reinforcement Learning Models
Deep reinforcement learning (DRL) has proven extremely useful in a large
variety of application domains. However, even successful DRL-based software can
exhibit highly undesirable behavior. This is due to DRL training being based on
maximizing a reward function, which typically captures general trends but
cannot precisely capture, or rule out, certain behaviors of the system. In this
paper, we propose a novel framework aimed at drastically reducing the
undesirable behavior of DRL-based software, while maintaining its excellent
performance. In addition, our framework can assist in providing engineers with
a comprehensible characterization of such undesirable behavior. Under the hood,
our approach is based on extracting decision tree classifiers from erroneous
state-action pairs, and then integrating these trees into the DRL training
loop, penalizing the system whenever it performs an error. We provide a
proof-of-concept implementation of our approach, and use it to evaluate the
technique on three significant case studies. We find that our approach can
extend existing frameworks in a straightforward manner, and incurs only a
slight overhead in training time. Further, it incurs only a very slight hit to
performance, or even in some cases - improves it, while significantly reducing
the frequency of undesirable behavior
A Survey on Explainable AI for 6G O-RAN: Architecture, Use Cases, Challenges and Research Directions
The recent O-RAN specifications promote the evolution of RAN architecture by
function disaggregation, adoption of open interfaces, and instantiation of a
hierarchical closed-loop control architecture managed by RAN Intelligent
Controllers (RICs) entities. This paves the road to novel data-driven network
management approaches based on programmable logic. Aided by Artificial
Intelligence (AI) and Machine Learning (ML), novel solutions targeting
traditionally unsolved RAN management issues can be devised. Nevertheless, the
adoption of such smart and autonomous systems is limited by the current
inability of human operators to understand the decision process of such AI/ML
solutions, affecting their trust in such novel tools. eXplainable AI (XAI) aims
at solving this issue, enabling human users to better understand and
effectively manage the emerging generation of artificially intelligent schemes,
reducing the human-to-machine barrier. In this survey, we provide a summary of
the XAI methods and metrics before studying their deployment over the O-RAN
Alliance RAN architecture along with its main building blocks. We then present
various use-cases and discuss the automation of XAI pipelines for O-RAN as well
as the underlying security aspects. We also review some projects/standards that
tackle this area. Finally, we identify different challenges and research
directions that may arise from the heavy adoption of AI/ML decision entities in
this context, focusing on how XAI can help to interpret, understand, and
improve trust in O-RAN operational networks.Comment: 33 pages, 13 figure
SliceOps: Explainable MLOps for Streamlined Automation-Native 6G Networks
Sixth-generation (6G) network slicing is the backbone of future
communications systems. It inaugurates the era of extreme ultra-reliable and
low-latency communication (xURLLC) and pervades the digitalization of the
various vertical immersive use cases. Since 6G inherently underpins artificial
intelligence (AI), we propose a systematic and standalone slice termed SliceOps
that is natively embedded in the 6G architecture, which gathers and manages the
whole AI lifecycle through monitoring, re-training, and deploying the machine
learning (ML) models as a service for the 6G slices. By leveraging machine
learning operations (MLOps) in conjunction with eXplainable AI (XAI), SliceOps
strives to cope with the opaqueness of black-box AI using explanation-guided
reinforcement learning (XRL) to fulfill transparency, trustworthiness, and
interpretability in the network slicing ecosystem. This article starts by
elaborating on the architectural and algorithmic aspects of SliceOps. Then, the
deployed cloud-native SliceOps working is exemplified via a latency-aware
resource allocation problem. The deep RL (DRL)-based SliceOps agents within
slices provide AI services aiming to allocate optimal radio resources and
impede service quality degradation. Simulation results demonstrate the
effectiveness of SliceOps-driven slicing. The article discusses afterward the
SliceOps challenges and limitations. Finally, the key open research directions
corresponding to the proposed approach are identified.Comment: 8 pages, 6 Figure
Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
Explainable Artificial Intelligence (XAI) is transforming the field of
Artificial Intelligence (AI) by enhancing the trust of end-users in machines.
As the number of connected devices keeps on growing, the Internet of Things
(IoT) market needs to be trustworthy for the end-users. However, existing
literature still lacks a systematic and comprehensive survey work on the use of
XAI for IoT. To bridge this lacking, in this paper, we address the XAI
frameworks with a focus on their characteristics and support for IoT. We
illustrate the widely-used XAI services for IoT applications, such as security
enhancement, Internet of Medical Things (IoMT), Industrial IoT (IIoT), and
Internet of City Things (IoCT). We also suggest the implementation choice of
XAI models over IoT systems in these applications with appropriate examples and
summarize the key inferences for future works. Moreover, we present the
cutting-edge development in edge XAI structures and the support of
sixth-generation (6G) communication services for IoT applications, along with
key inferences. In a nutshell, this paper constitutes the first holistic
compilation on the development of XAI-based frameworks tailored for the demands
of future IoT use cases.Comment: 29 pages, 7 figures, 2 tables. IEEE Open Journal of the
Communications Society (2022
Explainable and Safe Reinforcement Learning for Autonomous Air Mobility
Increasing traffic demands, higher levels of automation, and communication
enhancements provide novel design opportunities for future air traffic
controllers (ATCs). This article presents a novel deep reinforcement learning
(DRL) controller to aid conflict resolution for autonomous free flight.
Although DRL has achieved important advancements in this field, the existing
works pay little attention to the explainability and safety issues related to
DRL controllers, particularly the safety under adversarial attacks. To address
those two issues, we design a fully explainable DRL framework wherein we: 1)
decompose the coupled Q value learning model into a safety-awareness and
efficiency (reach the target) one; and 2) use information from surrounding
intruders as inputs, eliminating the needs of central controllers. In our
simulated experiments, we show that by decoupling the safety-awareness and
efficiency, we can exceed performance on free flight control tasks while
dramatically improving explainability on practical. In addition, the safety Q
learning module provides rich information about the safety situation of
environments. To study the safety under adversarial attacks, we additionally
propose an adversarial attack strategy that can impose both safety-oriented and
efficiency-oriented attacks. The adversarial aims to minimize safety/efficiency
by only attacking the agent at a few time steps. In the experiments, our attack
strategy increases as many collisions as the uniform attack (i.e., attacking at
every time step) by only attacking the agent four times less often, which
provide insights into the capabilities and restrictions of the DRL in future
ATC designs. The source code is publicly available at
https://github.com/WLeiiiii/Gym-ATC-Attack-Project
Recommended from our members
Explainable and Advisable Learning for Self-driving Vehicles
Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable - they should provide easy-to-interpret rationales for their behavior - so that passengers, insurance companies, law enforcement, developers, etc., can understand what triggered a particular behavior. Explanations may be triggered by the neural controller, namely introspective explanations, or informed by the neural controller's output, namely rationalizations. Our work has focused on the challenge of generating introspective explanations of deep models for self-driving vehicles. In Chapter 3, we begin by exploring the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network's output (steering control). In the first stage, we use a visual attention model to train a convolution network end-to-end from images to steering angle. The attention model highlights image regions that potentially influence the network's output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network's behavior. In Chapter 4, we add an attention-based video-to-text model to produce textual explanations of model actions, e.g. "the car slows down because the road is wet". The attention maps of controller and explanation model are aligned so that explanations are grounded in the parts of the scene that mattered to the controller. We explore two approaches to attention alignment, strong- and weak-alignment. These explainable systems represent an externalization of tacit knowledge. The network's opaque reasoning is simplified to a situation-specific dependence on a visible object in the image. This makes them brittle and potentially unsafe in situations that do not match training data. In Chapter 5, we propose to address this issue by augmenting training data with natural language advice from a human. Advice includes guidance about what to do and where to attend. We present the first step toward advice-giving, where we train an end-to-end vehicle controller that accepts advice. The controller adapts the way it attends to the scene (visual attention) and the control (steering and speed). Further, in Chapter 6, we propose a new approach that learns vehicle control with the help of long-term (global) human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (e.g. "I see a pedestrian crossing, so I stop"), and predict the controls, accordingly
- …