61,343 research outputs found
An efficient approach to model-based hierarchical reinforcement learning
National Research Foundation (NRF) Singapore under SMART and Future Mobility; Ministry of Education, Singapore under its Academic Research Funding Tier
DeepLine: AutoML Tool for Pipelines Generation using Deep Reinforcement Learning and Hierarchical Actions Filtering
Automatic machine learning (AutoML) is an area of research aimed at
automating machine learning (ML) activities that currently require human
experts. One of the most challenging tasks in this field is the automatic
generation of end-to-end ML pipelines: combining multiple types of ML
algorithms into a single architecture used for end-to-end analysis of
previously-unseen data. This task has two challenging aspects: the first is the
need to explore a large search space of algorithms and pipeline architectures.
The second challenge is the computational cost of training and evaluating
multiple pipelines. In this study we present DeepLine, a reinforcement learning
based approach for automatic pipeline generation. Our proposed approach
utilizes an efficient representation of the search space and leverages past
knowledge gained from previously-analyzed datasets to make the problem more
tractable. Additionally, we propose a novel hierarchical-actions algorithm that
serves as a plugin, mediating the environment-agent interaction in deep
reinforcement learning problems. The plugin significantly speeds up the
training process of our model. Evaluation on 56 datasets shows that DeepLine
outperforms state-of-the-art approaches both in accuracy and in computational
cost
Learning Representations in Model-Free Hierarchical Reinforcement Learning
Common approaches to Reinforcement Learning (RL) are seriously challenged by
large-scale applications involving huge state spaces and sparse delayed reward
feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address
this scalability issue by learning action selection policies at multiple levels
of temporal abstraction. Abstraction can be had by identifying a relatively
small set of states that are likely to be useful as subgoals, in concert with
the learning of corresponding skill policies to achieve those subgoals. Many
approaches to subgoal discovery in HRL depend on the analysis of a model of the
environment, but the need to learn such a model introduces its own problems of
scale. Once subgoals are identified, skills may be learned through intrinsic
motivation, introducing an internal reward signal marking subgoal attainment.
In this paper, we present a novel model-free method for subgoal discovery using
incremental unsupervised learning over a small memory of the most recent
experiences (trajectories) of the agent. When combined with an intrinsic
motivation learning mechanism, this method learns both subgoals and skills,
based on experiences in the environment. Thus, we offer an original approach to
HRL that does not require the acquisition of a model of the environment,
suitable for large-scale applications. We demonstrate the efficiency of our
method on two RL problems with sparse delayed feedback: a variant of the rooms
environment and the first screen of the ATARI 2600 Montezuma's Revenge game
- …