4,216 research outputs found
Reinforcement Learning With Temporal Logic Rewards
Reinforcement learning (RL) depends critically on the choice of reward
functions used to capture the de- sired behavior and constraints of a robot.
Usually, these are handcrafted by a expert designer and represent heuristics
for relatively simple tasks. Real world applications typically involve more
complex tasks with rich temporal and logical structure. In this paper we take
advantage of the expressive power of temporal logic (TL) to specify complex
rules the robot should follow, and incorporate domain knowledge into learning.
We propose Truncated Linear Temporal Logic (TLTL) as specifications language,
that is arguably well suited for the robotics applications, together with
quantitative semantics, i.e., robustness degree. We propose a RL approach to
learn tasks expressed as TLTL formulae that uses their associated robustness
degree as reward functions, instead of the manually crafted heuristics trying
to capture the same specifications. We show in simulated trials that learning
is faster and policies obtained using the proposed approach outperform the ones
learned using heuristic rewards in terms of the robustness degree, i.e., how
well the tasks are satisfied. Furthermore, we demonstrate the proposed RL
approach in a toast-placing task learned by a Baxter robot
The 1990 progress report and future plans
This document describes the progress and plans of the Artificial Intelligence Research Branch (RIA) at ARC in 1990. Activities span a range from basic scientific research to engineering development and to fielded NASA applications, particularly those applications that are enabled by basic research carried out at RIA. Work is conducted in-house and through collaborative partners in academia and industry. Our major focus is on a limited number of research themes with a dual commitment to technical excellence and proven applicability to NASA short, medium, and long-term problems. RIA acts as the Agency's lead organization for research aspects of artificial intelligence, working closely with a second research laboratory at JPL and AI applications groups at all NASA centers
Automata learning algorithms and processes for providing more complete systems requirements specification by scenario generation, CSP-based syntax-oriented model construction, and R2D2C system requirements transformation
Systems, methods and apparatus are provided through which in some embodiments, automata learning algorithms and techniques are implemented to generate a more complete set of scenarios for requirements based programming. More specifically, a CSP-based, syntax-oriented model construction, which requires the support of a theorem prover, is complemented by model extrapolation, via automata learning. This may support the systematic completion of the requirements, the nature of the requirement being partial, which provides focus on the most prominent scenarios. This may generalize requirement skeletons by extrapolation and may indicate by way of automatically generated traces where the requirement specification is too loose and additional information is required
Explaining Aha! moments in artificial agents through IKE-XAI: Implicit Knowledge Extraction for eXplainable AI
During the learning process, a child develops a mental representation of the task he or she is learning.
A Machine Learning algorithm develops also a latent representation of the task it learns. We investigate
the development of the knowledge construction of an artificial agent through the analysis of its
behavior, i.e., its sequences of moves while learning to perform the Tower of Hanoï (TOH) task. The TOH
is a well-known task in experimental contexts to study the problem-solving processes and one of the
fundamental processes of children’s knowledge construction about their world. We position ourselves
in the field of explainable reinforcement learning for developmental robotics, at the crossroads of
cognitive modeling and explainable AI. Our main contribution proposes a 3-step methodology named
Implicit Knowledge Extraction with eXplainable Artificial Intelligence (IKE-XAI) to extract the implicit
knowledge, in form of an automaton, encoded by an artificial agent during its learning. We showcase
this technique to solve and explain the TOH task when researchers have only access to moves that
represent observational behavior as in human–machine interaction. Therefore, to extract the agent
acquired knowledge at different stages of its training, our approach combines: first, a Q-learning
agent that learns to perform the TOH task; second, a trained recurrent neural network that encodes
an implicit representation of the TOH task; and third, an XAI process using a post-hoc implicit rule
extraction algorithm to extract finite state automata. We propose using graph representations as visual
and explicit explanations of the behavior of the Q-learning agent. Our experiments show that the IKEXAI
approach helps understanding the development of the Q-learning agent behavior by providing
a global explanation of its knowledge evolution during learning. IKE-XAI also allows researchers to
identify the agent’s Aha! moment by determining from what moment the knowledge representation
stabilizes and the agent no longer learns.Region BretagneEuropean Union via the FEDER programSpanish Government Juan de la Cierva Incorporacion - MCIN/AEI IJC2019-039152-IGoogle Research Scholar Gran
- …