18 research outputs found
Combined Reinforcement Learning via Abstract Representations
In the quest for efficient and robust reinforcement learning methods, both
model-free and model-based approaches offer advantages. In this paper we
propose a new way of explicitly bridging both approaches via a shared
low-dimensional learned encoding of the environment, meant to capture
summarizing abstractions. We show that the modularity brought by this approach
leads to good generalization while being computationally efficient, with
planning happening in a smaller latent state space. In addition, this approach
recovers a sufficient low-dimensional representation of the environment, which
opens up new strategies for interpretable AI, exploration and transfer
learning.Comment: Accepted to the Thirty-Third AAAI Conference On Artificial
Intelligence, 201
Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces
Autonomous robots require high degrees of cognitive and motoric intelligence
to come into our everyday life. In non-structured environments and in the
presence of uncertainties, such degrees of intelligence are not easy to obtain.
Reinforcement learning algorithms have proven to be capable of solving
complicated robotics tasks in an end-to-end fashion without any need for
hand-crafted features or policies. Especially in the context of robotics, in
which the cost of real-world data is usually extremely high, reinforcement
learning solutions achieving high sample efficiency are needed. In this paper,
we propose a framework combining the learning of a low-dimensional state
representation, from high-dimensional observations coming from the robot's raw
sensory readings, with the learning of the optimal policy, given the learned
state representation. We evaluate our framework in the context of mobile robot
navigation in the case of continuous state and action spaces. Moreover, we
study the problem of transferring what learned in the simulated virtual
environment to the real robot without further retraining using real-world data
in the presence of visual and depth distractors, such as lighting changes and
moving obstacles.Comment: Paper Accepted at IROS2021. This work has been submitted to the IEEE
for possible publication. Copyright may be transferred without notice, after
which this version may no longer be accessibl
Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Tasks
With the recent prevalence of reinforcement learning (RL), there have been
tremendous interests in utilizing RL for ads allocation in recommendation
platforms (e.g., e-commerce and news feed sites). For better performance,
recent RL-based ads allocation agent makes decisions based on representations
of list-wise item arrangement. This results in a high-dimensional state-action
space, which makes it difficult to learn an efficient and generalizable
list-wise representation. To address this problem, we propose a novel algorithm
to learn a better representation by leveraging task-specific signals on Meituan
food delivery platform. Specifically, we propose three different types of
auxiliary tasks that are based on reconstruction, prediction, and contrastive
learning respectively. We conduct extensive offline experiments on the
effectiveness of these auxiliary tasks and test our method on real-world food
delivery platform. The experimental results show that our method can learn
better list-wise representations and achieve higher revenue for the platform.Comment: arXiv admin note: text overlap with arXiv:2109.04353,
arXiv:2204.0037