4,121 research outputs found
Distral: Robust Multitask Reinforcement Learning
Most deep reinforcement learning algorithms are data inefficient in complex
and rich environments, limiting their applicability to many scenarios. One
direction for improving data efficiency is multitask learning with shared
neural network parameters, where efficiency may be improved through transfer
across related tasks. In practice, however, this is not usually observed,
because gradients from different tasks can interfere negatively, making
learning unstable and sometimes even less data efficient. Another issue is the
different reward schemes between tasks, which can easily lead to one task
dominating the learning of a shared model. We propose a new approach for joint
training of multiple tasks, which we refer to as Distral (Distill & transfer
learning). Instead of sharing parameters between the different workers, we
propose to share a "distilled" policy that captures common behaviour across
tasks. Each worker is trained to solve its own task while constrained to stay
close to the shared policy, while the shared policy is trained by distillation
to be the centroid of all task policies. Both aspects of the learning process
are derived by optimizing a joint objective function. We show that our approach
supports efficient transfer on complex 3D environments, outperforming several
related methods. Moreover, the proposed learning process is more robust and
more stable---attributes that are critical in deep reinforcement learning
Revisiting Pretraining Objectives for Tabular Deep Learning
Recent deep learning models for tabular data currently compete with the
traditional ML models based on decision trees (GBDT). Unlike GBDT, deep models
can additionally benefit from pretraining, which is a workhorse of DL for
vision and NLP. For tabular problems, several pretraining methods were
proposed, but it is not entirely clear if pretraining provides consistent
noticeable improvements and what method should be used, since the methods are
often not compared to each other or comparison is limited to the simplest MLP
architectures.
In this work, we aim to identify the best practices to pretrain tabular DL
models that can be universally applied to different datasets and architectures.
Among our findings, we show that using the object target labels during the
pretraining stage is beneficial for the downstream performance and advocate
several target-aware pretraining objectives. Overall, our experiments
demonstrate that properly performed pretraining significantly increases the
performance of tabular DL models, which often leads to their superiority over
GBDTs.Comment: Code: https://github.com/puhsu/tabular-dl-pretrain-objective
Count-Based Exploration with the Successor Representation
In this paper we introduce a simple approach for exploration in reinforcement
learning (RL) that allows us to develop theoretically justified algorithms in
the tabular case but that is also extendable to settings where function
approximation is required. Our approach is based on the successor
representation (SR), which was originally introduced as a representation
defining state generalization by the similarity of successor states. Here we
show that the norm of the SR, while it is being learned, can be used as a
reward bonus to incentivize exploration. In order to better understand this
transient behavior of the norm of the SR we introduce the substochastic
successor representation (SSR) and we show that it implicitly counts the number
of times each state (or feature) has been observed. We use this result to
introduce an algorithm that performs as well as some theoretically
sample-efficient approaches. Finally, we extend these ideas to a deep RL
algorithm and show that it achieves state-of-the-art performance in Atari 2600
games when in a low sample-complexity regime.Comment: This paper appears in the Proceedings of the 34th AAAI Conference on
Artificial Intelligence (AAAI 2020
A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning
Academic tabular benchmarks often contain small sets of curated features. In
contrast, data scientists typically collect as many features as possible into
their datasets, and even engineer new features from existing ones. To prevent
overfitting in subsequent downstream modeling, practitioners commonly use
automated feature selection methods that identify a reduced subset of
informative features. Existing benchmarks for tabular feature selection
consider classical downstream models, toy synthetic datasets, or do not
evaluate feature selectors on the basis of downstream performance. Motivated by
the increasing popularity of tabular deep learning, we construct a challenging
feature selection benchmark evaluated on downstream neural networks including
transformers, using real datasets and multiple methods for generating
extraneous features. We also propose an input-gradient-based analogue of Lasso
for neural networks that outperforms classical feature selection methods on
challenging problems such as selecting from corrupted or second-order features
Tabular Machine Learning Methods for Predicting Gas Turbine Emissions
The work presented here received funding from EPSRC (EP/W522089/1) and Siemens Energy Industrial Turbomachinery Ltd. as part of the iCASE EPSRC PhD studentship ”Predictive Emission Monitoring Systems for Gas Turbines”.Preprin
Tabular Machine Learning Methods for Predicting Gas Turbine Emissions
The work presented here received funding from EPSRC (EP/W522089/1) and Siemens Energy Industrial Turbomachinery Ltd. as part of the iCASE EPSRC PhD studentship “Predictive Emission Monitoring Systems for Gas Turbines”.Peer reviewedPublisher PD
Fine-Tuning the Retrieval Mechanism for Tabular Deep Learning
While interests in tabular deep learning has significantly grown,
conventional tree-based models still outperform deep learning methods. To
narrow this performance gap, we explore the innovative retrieval mechanism, a
methodology that allows neural networks to refer to other data points while
making predictions. Our experiments reveal that retrieval-based training,
especially when fine-tuning the pretrained TabPFN model, notably surpasses
existing methods. Moreover, the extensive pretraining plays a crucial role to
enhance the performance of the model. These insights imply that blending the
retrieval mechanism with pretraining and transfer learning schemes offers
considerable potential for advancing the field of tabular deep learning.Comment: Table Representation Learning Workshop at NeurIPS 202
- …