Combining Experience Replay with Exploration by Random Network Distillation

Our work is a simple extension of the paper "Exploration by Random Network Distillation". More in detail, we show how to efficiently combine Intrinsic Rewards with Experience Replay in order to achieve more efficient and robust exploration (with respect to PPO/RND) and consequently better results in terms of agent performances and sample efficiency. We are able to do it by using a new technique named Prioritized Oversampled Experience Replay (POER), that has been built upon the definition of what is the important experience useful to replay. Finally, we evaluate our technique on the famous Atari game Montezuma's Revenge and some other hard exploration Atari games.Comment: 8 pages, 6 figures, accepted as full-paper at IEEE Conference on Games (CoG) 201

Deep Reinforcement Learning and sub-problem decomposition using Hierarchical Architectures in partially observable environments

Reinforcement Learning (RL) is based on the Markov Decision Process (MDP) framework, but not all the problems of interest can be modeled with MDPs because some of them have non-markovian temporal dependencies. To handle them, one of the solutions proposed in literature is Hierarchical Reinforcement Learning (HRL). HRL takes inspiration from hierarchical planning in artificial intelligence literature and it is an emerging sub-discipline for RL, in which RL methods are augmented with some kind of prior knowledge about the high-level structure of behavior in order to decompose the underlying problem into simpler sub-problems. The high-level goal of our thesis is to investigate the advantages that a HRL approach may have over a simple RL approach. Thus, we study problems of interest (rarely tackled by mean of RL) like Sentiment Analysis, Rogue and Car Controller, showing how the ability of RL algorithms to solve them in a partially observable environment is affected by using (or not) generic hierarchical architectures based on RL algorithms of the Actor-Critic family. Remarkably, we claim that especially our work in Sentiment Analysis is very innovative for RL, resulting in state-of-the-art performances; as far as the author knows, Reinforcement Learning approach is only rarely applied to the domain of computational linguistic and sentiment analysis. Furthermore, our work on the famous video-game Rogue is probably the first example of Deep RL architecture able to explore Rogue dungeons and fight against its monsters achieving a success rate of more than 75% on the first game level. While our work on Car Controller allowed us to make some interesting considerations on the nature of some components of the policy gradient equation

How to Quantify the Degree of Explainability: Experiments and Practical Implications

Explainable AI was born as a pathway to allow humans to explore and understand the inner working of complex systems. Though, establishing what is an explanation and objectively evaluating explainability, are not trivial tasks. With this paper, we present a new model-agnostic metric to measure the Degree of Explainability of (correct) information in an objective way, exploiting a specific theoretical model from Ordinary Language Philosophy called the Achinstein’s Theory of Explanations, implemented with an algorithm relying on deep language models for knowledge graph extraction and information retrieval. In order to understand whether this metric is actually behaving as explainability is expected to, we have devised an experiment on two realistic Explainable AI-based systems for healthcare and finance, using famous AI technology including Artificial Neural Networks and TreeSHAP. The results we obtained suggest that our proposed metric for measuring the Degree of Explainability is robust on several scenario

Nonlinear differential equations having non-sign-definite weights

In the present PhD thesis we deal with the study of the existence, multiplicity and complex behaviors of solutions for some classes of boundary value problems associated with second order nonlinear ordinary differential equations of the form $u''+f(u)u'+g(t,u)=s,$ or $u''+g(t,u)=0,$ $t\in I$, where $I$ is a bounded interval, $f\colon\mathbb{R}\to\mathbb{R}$ is continuous, $s\in\mathbb{R}$ and $g: I\times \mathbb{R}\to\mathbb{R}$ is a perturbation term characterizing the problems. The results carried out in this dissertation are mainly based on dynamical and topological approaches. The issues we address have arisen in the field of partial differential equations. For this reason, we do not treat only the case of ordinary differential equations, but also we take advantage of some results achieved in the one dimensional setting to give applications to nonlinear boundary value problems associated with partial differential equations. In the first part of the thesis, we are interested on a problem suggested by Antonio Ambrosetti in ``Observations on global inversion theorems'' (2011). In more detail, we deal with a periodic boundary value problem associated with the first differential equation where the perturbation term is given by $g(t,u):=a(t)\phi(u)-p(t)$. We assume that $a,$ $p\in L^{\infty}(I)$ and $\phi\colon\mathbb{R}\to\mathbb{R}$ is a continuous function satisfying $\lim_{|\xi|\to\infty}\phi(\xi)=+\infty$. In this context, if the weight term $a(t)$ is such that $a(t)\geq 0$ for a.e. $t\in I$ and $\int_{I}a(t)\,dt>0$, we generalize the result of multiplicity of solutions given by Fabry, Mawhin and Nakashama in ``A multiplicity result for periodic solutions of forced nonlinear second order ordinary differential equations'' (1986). We extend this kind of improvement also to more general nonlinear terms under local coercivity conditions. In this framework, we also treat in the same spirit Neumann problems associated with second order ordinary differential equations and periodic problems associated with first order ones. Furthermore, we face the classical case of a periodic Ambrosetti-Prodi problem with a weight term $a(t)$ which is constant and positive. Here, considering in the second differential equation a nonlinearity $g(t,u):=\phi(u)-h(t)$, we provide several conditions on the nonlinearity and the perturbative term that ensure the presence of complex behaviors for the solutions of the associated $T$-periodic problem. We also compare these outcomes with the result of stability carried out by Ortega in ``Stability of a periodic problem of Ambrosetti-Prodi type'' (1990). The case with damping term is discussed as well. In the second part of this work, we solve a conjecture by Yuan Lou and Thomas Nagylaki stated in ``A semilinear parabolic system for migration and selection in population genetics'' (2002). The problem refers to the number of positive solutions for Neumann boundary value problems associated with the second differential equation when the perturbation term is given by $g(t,u):=\lambda w(t)\psi(u)$ with $\lambda>0$, $w\in L^{\infty}(I)$ a sign-changing weight term such that $\int_{I}w(t)\,dt<0$ and $\psi\colon[0,1]\to[0,\infty[$ a non-concave continuous function satisfying $\psi(0)=0=\psi(1)$ and such that the map $\xi\mapsto \psi(\xi)/\xi$ is monotone decreasing. In addition to this outcome, other new results of multiplicity of positive solutions are presented as well, for both Neumann or Dirichlet boundary value problems, by means of a particular choice of indefinite weight terms $w(t)$ and different positive nonlinear terms $\psi(u)$ defined on the interval $[0,1]$ or on the positive real semi-axis $[0,+\infty[$

Remarks on Dirichlet problems with sub linear growth at infinity

We present some existence and multiplicity results for positive solutions to the Dirichlet problem associated with; under suitable conditions on the nonlinearity g(u)and thew eight function a(x): The assumptions considered are related to classical theorems about positive solutions to a sublinear elliptic equation due to Brezis-Oswald and Brown-Hess

Generating User-Centred Explanations via Illocutionary Question Answering: From Philosophy to Interfaces

We propose a new method for generating explanations with Artificial Intelligence (AI) and a tool to test its expressive power within a user interface. In order to bridge the gap between philosophy and human-computer interfaces, we show a new approach for the generation of interactive explanations based on a sophisticated pipeline of AI algorithms for structuring natural language documents into knowledge graphs, answering questions effectively and satisfactorily. With this work, we aim to prove that the philosophical theory of explanations presented by Achinstein can be actually adapted for being implemented into a concrete software application, as an interactive and illocutionary process of answering questions. Specifically, our contribution is an approach to frame illocution in a computer-friendly way, to achieve user-centrality with statistical question answering. Indeed, we frame the illocution of an explanatory process as that mechanism responsible for anticipating the needs of the explainee in the form of unposed, implicit, archetypal questions, hence improving the user-centrality of the underlying explanatory process. Therefore, we hypothesise that if an explanatory process is an illocutionary act of providing content-giving answers to questions, and illocution is as we defined it, the more explicit and implicit questions can be answered by an explanatory tool, the more usable (as per ISO 9241-210) its explanations. We tested our hypothesis with a user-study involving more than 60 participants, on two XAI-based systems, one for credit approval (finance) and one for heart disease prediction (healthcare). The results showed that increasing the illocutionary power of an explanatory tool can produce statistically significant improvements (hence with a P value lower than .05) on effectiveness. This, combined with a visible alignment between the increments in effectiveness and satisfaction, suggests that our understanding of illocution can be correct, giving evidence in favour of our theory

Explanatory artificial intelligence (YAI): human-centered explanations of explainable AI and complex data

In this paper we introduce a new class of software tools engaged in delivering successful explanations of complex processes on top of basic Explainable AI (XAI) software systems. These tools, that we call cumulatively Explanatory AI (YAI) systems, enhance the quality of the basic output of a XAI by adopting a user-centred approach to explanation that can cater to the individual needs of the explainees with measurable improvements in usability. Our approach is based on Achinstein’s theory of explanations, where explaining is an illocutionary (i.e., broad yet pertinent and deliberate) act of pragmatically answering a question. Accordingly, user-centrality enters in the equation by considering that the overall amount of information generated by answering all questions can rapidly become overwhelming and that individual users may perceive the need to explore just a few of them. In this paper, we give the theoretical foundations of YAI, formally defining a user-centred explanatory tool and the space of all possible explanations, or explanatory space, generated by it. To this end, we frame the explanatory space as an hypergraph of knowledge and we identify a set of heuristics and properties that can help approximating a decomposition of it into a tree-like representation for efficient and user-centred explanation retrieval. Finally, we provide some old and new empirical results to support our theory, showing that explanations are more than textual or visual presentations of the sole information provided by a XAI

Dolcher fixed point theorem and its connections with recent developments on compressive/expansive maps

Elisa Sovrano and Fabio Zanolin, "Dolcher fixed point theorem and its connections with recent developments on compressive/expansive maps", in: Rendiconti dell’Istituto di Matematica dell’Università di Trieste. An International Journal of Mathematics, 46 (2014), pp.101-121In 1948 Mario Dolcher proposed an expansive version of the Brouwer fixed point theorem for planar maps. In this article we reconsider Dolcher's result in connection with some properties, such as covering relations, which appear in the study of chaotic dynamics

Foreseeing the Impact of the Proposed AI Act on the Sustainability and Safety of Critical Infrastructures

The AI Act has been recently proposed by the European Commission to regulate the use of AI in the EU, especially on high-risk applications, i.e. systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity. On the other hand, IEC 61508, one of the most adopted international standards for safety-critical electronic components, seem to mostly forbid the use of AI in such systems. Given this conflict between IEC 61508 and the proposed AI Act, also stressed by the fact that IEC 61508 is not an harmonised European standard, with the present paper we study and analyse what is going to happen to industry after the entry into force of the AI Act. In particular, we focus on how the proposed AI Act might positively impact on the sustainability of critical infrastructures by allowing the use of AI on an industry where it was previously forbidden. To do so, we provide several examples of AI-based solutions falling under the umbrella of IEC 61508 that might have a positive impact on sustainability in alignment with the current long-term goals of the EU and the Sustainable Development Goals of the United Nations, i.e., affordable and clean energy, sustainable cities and communities