88 research outputs found
Master-slave Deep Architecture for Top-K Multi-armed Bandits with Non-linear Bandit Feedback and Diversity Constraints
We propose a novel master-slave architecture to solve the top-
combinatorial multi-armed bandits problem with non-linear bandit feedback and
diversity constraints, which, to the best of our knowledge, is the first
combinatorial bandits setting considering diversity constraints under bandit
feedback. Specifically, to efficiently explore the combinatorial and
constrained action space, we introduce six slave models with distinguished
merits to generate diversified samples well balancing rewards and constraints
as well as efficiency. Moreover, we propose teacher learning based optimization
and the policy co-training technique to boost the performance of the multiple
slave models. The master model then collects the elite samples provided by the
slave models and selects the best sample estimated by a neural contextual
UCB-based network to make a decision with a trade-off between exploration and
exploitation. Thanks to the elaborate design of slave models, the co-training
mechanism among slave models, and the novel interactions between the master and
slave models, our approach significantly surpasses existing state-of-the-art
algorithms in both synthetic and real datasets for recommendation tasks. The
code is available at:
\url{https://github.com/huanghanchi/Master-slave-Algorithm-for-Top-K-Bandits}.Comment: IEEE Transactions on Neural Networks and Learning System
Hierarchical Graph Transformer with Adaptive Node Sampling
The Transformer architecture has achieved remarkable success in a number of
domains including natural language processing and computer vision. However,
when it comes to graph-structured data, transformers have not achieved
competitive performance, especially on large graphs. In this paper, we identify
the main deficiencies of current graph transformers:(1) Existing node sampling
strategies in Graph Transformers are agnostic to the graph characteristics and
the training process. (2) Most sampling strategies only focus on local
neighbors and neglect the long-range dependencies in the graph. We conduct
experimental investigations on synthetic datasets to show that existing
sampling strategies are sub-optimal. To tackle the aforementioned problems, we
formulate the optimization strategies of node sampling in Graph Transformer as
an adversary bandit problem, where the rewards are related to the attention
weights and can vary in the training procedure. Meanwhile, we propose a
hierarchical attention scheme with graph coarsening to capture the long-range
interactions while reducing computational complexity. Finally, we conduct
extensive experiments on real-world datasets to demonstrate the superiority of
our method over existing graph transformers and popular GNNs.Comment: Accepted by NeurIPS 202
Cakewalk Sampling
We study the task of finding good local optima in combinatorial optimization
problems. Although combinatorial optimization is NP-hard in general, locally
optimal solutions are frequently used in practice. Local search methods however
typically converge to a limited set of optima that depend on their
initialization. Sampling methods on the other hand can access any valid
solution, and thus can be used either directly or alongside methods of the
former type as a way for finding good local optima. Since the effectiveness of
this strategy depends on the sampling distribution, we derive a robust learning
algorithm that adapts sampling distributions towards good local optima of
arbitrary objective functions. As a first use case, we empirically study the
efficiency in which sampling methods can recover locally maximal cliques in
undirected graphs. Not only do we show how our adaptive sampler outperforms
related methods, we also show how it can even approach the performance of
established clique algorithms. As a second use case, we consider how greedy
algorithms can be combined with our adaptive sampler, and we demonstrate how
this leads to superior performance in k-medoid clustering. Together, these
findings suggest that our adaptive sampler can provide an effective strategy to
combinatorial optimization problems that arise in practice.Comment: Accepted as a conference paper by AAAI-2020 (oral presentation
Active model learning and diverse action sampling for task and motion planning
The objective of this work is to augment the basic abilities of a robot by
learning to use new sensorimotor primitives to enable the solution of complex
long-horizon problems. Solving long-horizon problems in complex domains
requires flexible generative planning that can combine primitive abilities in
novel combinations to solve problems as they arise in the world. In order to
plan to combine primitive actions, we must have models of the preconditions and
effects of those actions: under what circumstances will executing this
primitive achieve some particular effect in the world?
We use, and develop novel improvements on, state-of-the-art methods for
active learning and sampling. We use Gaussian process methods for learning the
conditions of operator effectiveness from small numbers of expensive training
examples collected by experimentation on a robot. We develop adaptive sampling
methods for generating diverse elements of continuous sets (such as robot
configurations and object poses) during planning for solving a new task, so
that planning is as efficient as possible. We demonstrate these methods in an
integrated system, combining newly learned models with an efficient
continuous-space robot task and motion planner to learn to solve long horizon
problems more efficiently than was previously possible.Comment: Proceedings of the 2018 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), Madrid, Spain.
https://www.youtube.com/playlist?list=PLoWhBFPMfSzDbc8CYelsbHZa1d3uz-W_
Graph Generative Model for Benchmarking Graph Neural Networks
As the field of Graph Neural Networks (GNN) continues to grow, it experiences
a corresponding increase in the need for large, real-world datasets to train
and test new GNN models on challenging, realistic problems. Unfortunately, such
graph datasets are often generated from online, highly privacy-restricted
ecosystems, which makes research and development on these datasets hard, if not
impossible. This greatly reduces the amount of benchmark graphs available to
researchers, causing the field to rely only on a handful of publicly-available
datasets. To address this problem, we introduce a novel graph generative model,
Computation Graph Transformer (CGT) that learns and reproduces the distribution
of real-world graphs in a privacy-controlled way. More specifically, CGT (1)
generates effective benchmark graphs on which GNNs show similar task
performance as on the source graphs, (2) scales to process large-scale graphs,
(3) incorporates off-the-shelf privacy modules to guarantee end-user privacy of
the generated graph. Extensive experiments across a vast body of graph
generative models show that only our model can successfully generate
privacy-controlled, synthetic substitutes of large-scale real-world graphs that
can be effectively used to benchmark GNN models
- …