158 research outputs found
Task Transfer by Preference-Based Cost Learning
The goal of task transfer in reinforcement learning is migrating the action
policy of an agent to the target task from the source task. Given their
successes on robotic action planning, current methods mostly rely on two
requirements: exactly-relevant expert demonstrations or the explicitly-coded
cost function on target task, both of which, however, are inconvenient to
obtain in practice. In this paper, we relax these two strong conditions by
developing a novel task transfer framework where the expert preference is
applied as a guidance. In particular, we alternate the following two steps:
Firstly, letting experts apply pre-defined preference rules to select related
expert demonstrates for the target task. Secondly, based on the selection
result, we learn the target cost function and trajectory distribution
simultaneously via enhanced Adversarial MaxEnt IRL and generate more
trajectories by the learned target distribution for the next preference
selection. The theoretical analysis on the distribution learning and
convergence of the proposed algorithm are provided. Extensive simulations on
several benchmarks have been conducted for further verifying the effectiveness
of the proposed method.Comment: Accepted to AAAI 2019. Mingxuan Jing and Xiaojian Ma contributed
equally to this wor
Tackling Over-Smoothing for General Graph Convolutional Networks
Increasing the depth of GCN, which is expected to permit more expressivity,
is shown to incur performance detriment especially on node classification. The
main cause of this lies in over-smoothing. The over-smoothing issue drives the
output of GCN towards a space that contains limited distinguished information
among nodes, leading to poor expressivity. Several works on refining the
architecture of deep GCN have been proposed, but it is still unknown in theory
whether or not these refinements are able to relieve over-smoothing. In this
paper, we first theoretically analyze how general GCNs act with the increase in
depth, including generic GCN, GCN with bias, ResGCN, and APPNP. We find that
all these models are characterized by a universal process: all nodes converging
to a cuboid. Upon this theorem, we propose DropEdge to alleviate over-smoothing
by randomly removing a certain number of edges at each training epoch.
Theoretically, DropEdge either reduces the convergence speed of over-smoothing
or relieves the information loss caused by dimension collapse. Experimental
evaluations on simulated dataset have visualized the difference in
over-smoothing between different GCNs. Moreover, extensive experiments on
several real benchmarks support that DropEdge consistently improves the
performance on a variety of both shallow and deep GCNs.Comment: Submitted to TPAMI, 15 page
Constant-size ciphertexts in threshold attribute-based encryption without dummy attributes
Attribute-based encryption (ABE) is an augmentation of public key encryption that allows users to encrypt and decrypt messages based on users’ attributes. In a ( t, s ) threshold ABE, users who can decrypt a ciphertext must hold at least t attributes among the s attributes specified by the encryptor. At PKC 2010, Herranz, Laguillaumie and Rà fols proposed the first threshold ABE with constant-size ciphertexts. In order to ensure the encryptor can flexibly select the attribute set and a threshold value, they use dummy attributes to satisfy the decryption requirement. The advantage of their scheme is that any addition or removal of the attributes will not require any change to users’ private keys or public parameters. Unfortunately, the need for dummy attributes makes their scheme inefficient, since the computational cost of encryption is linear to the size of selected attribute set and dummy attribute set. In this work, we improve Herranz et al.’s work, and propose a new threshold ABE scheme which does not use any dummy attribute . Our scheme not only retains the nice feature of Herranz et al.’s scheme, but also offers two improvements in comparison to the previous work. Firstly, the computational costs of encryption and decryption are only linear in the size of the selected attribute set. Secondly, without any dummy attribute, most of the computations can be conducted without the knowledge of the threshold t . Hence, threshold change in the encryption phase does not require complete recomputation of the ciphertext
Subequivariant Graph Reinforcement Learning in 3D Environments
Learning a shared policy that guides the locomotion of different agents is of
core interest in Reinforcement Learning (RL), which leads to the study of
morphology-agnostic RL. However, existing benchmarks are highly restrictive in
the choice of starting point and target point, constraining the movement of the
agents within 2D space. In this work, we propose a novel setup for
morphology-agnostic RL, dubbed Subequivariant Graph RL in 3D environments
(3D-SGRL). Specifically, we first introduce a new set of more practical yet
challenging benchmarks in 3D space that allows the agent to have full
Degree-of-Freedoms to explore in arbitrary directions starting from arbitrary
configurations. Moreover, to optimize the policy over the enlarged state-action
space, we propose to inject geometric symmetry, i.e., subequivariance, into the
modeling of the policy and Q-function such that the policy can generalize to
all directions, improving exploration efficiency. This goal is achieved by a
novel SubEquivariant Transformer (SET) that permits expressive message
exchange. Finally, we evaluate the proposed method on the proposed benchmarks,
where our method consistently and significantly outperforms existing approaches
on single-task, multi-task, and zero-shot generalization scenarios. Extensive
ablations are also conducted to verify our design. Code and videos are
available on our project page: https://alpc91.github.io/SGRL/.Comment: ICML 2023 Ora
Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction
Multimodal fusion and multitask learning are two vital topics in machine
learning. Despite the fruitful progress, existing methods for both problems are
still brittle to the same challenge -- it remains dilemmatic to integrate the
common information across modalities (resp. tasks) meanwhile preserving the
specific patterns of each modality (resp. task). Besides, while they are
actually closely related to each other, multimodal fusion and multitask
learning are rarely explored within the same methodological framework before.
In this paper, we propose Channel-Exchanging-Network (CEN) which is
self-adaptive, parameter-free, and more importantly, applicable for both
multimodal fusion and multitask learning. At its core, CEN dynamically
exchanges channels between subnetworks of different modalities. Specifically,
the channel exchanging process is self-guided by individual channel importance
that is measured by the magnitude of Batch-Normalization (BN) scaling factor
during training. For the application of dense image prediction, the validity of
CEN is tested by four different scenarios: multimodal fusion, cycle multimodal
fusion, multitask learning, and multimodal multitask learning. Extensive
experiments on semantic segmentation via RGB-D data and image translation
through multi-domain input verify the effectiveness of our CEN compared to
current state-of-the-art methods. Detailed ablation studies have also been
carried out, which provably affirm the advantage of each component we propose.Comment: 18 pages. arXiv admin note: substantial text overlap with
arXiv:2011.0500
SE-Bridge: Speech Enhancement with Consistent Brownian Bridge
We propose SE-Bridge, a novel method for speech enhancement (SE). After
recently applying the diffusion models to speech enhancement, we can achieve
speech enhancement by solving a stochastic differential equation (SDE). Each
SDE corresponds to a probabilistic flow ordinary differential equation
(PF-ODE), and the trajectory of the PF-ODE solution consists of the speech
states at different moments. Our approach is based on consistency model that
ensure any speech states on the same PF-ODE trajectory, correspond to the same
initial state. By integrating the Brownian Bridge process, the model is able to
generate high-intelligibility speech samples without adversarial training. This
is the first attempt that applies the consistency models to SE task, achieving
state-of-the-art results in several metrics while saving 15 x the time required
for sampling compared to the diffusion-based baseline. Our experiments on
multiple datasets demonstrate the effectiveness of SE-Bridge in SE.
Furthermore, we show through extensive experiments on downstream tasks,
including Automatic Speech Recognition (ASR) and Speaker Verification (SV),
that SE-Bridge can effectively support multiple downstream tasks
- …