1,971 research outputs found
Pre-trained Word Embeddings for Goal-conditional Transfer Learning in Reinforcement Learning
Reinforcement learning (RL) algorithms typically start tabula rasa, without
any prior knowledge of the environment, and without any prior skills. This
however often leads to low sample efficiency, requiring a large amount of
interaction with the environment. This is especially true in a lifelong
learning setting, in which the agent needs to continually extend its
capabilities. In this paper, we examine how a pre-trained task-independent
language model can make a goal-conditional RL agent more sample efficient. We
do this by facilitating transfer learning between different related tasks. We
experimentally demonstrate our approach on a set of object navigation tasks.Comment: Paper accepted to the ICML 2020 Language in Reinforcement Learning
(LaReL) Worksho
Few-shot classification in Named Entity Recognition Task
For many natural language processing (NLP) tasks the amount of annotated data
is limited. This urges a need to apply semi-supervised learning techniques,
such as transfer learning or meta-learning. In this work we tackle Named Entity
Recognition (NER) task using Prototypical Network - a metric learning
technique. It learns intermediate representations of words which cluster well
into named entity classes. This property of the model allows classifying words
with extremely limited number of training examples, and can potentially be used
as a zero-shot learning method. By coupling this technique with transfer
learning we achieve well-performing classifiers trained on only 20 instances of
a target class.Comment: In proceedings of the 34th ACM/SIGAPP Symposium on Applied Computin
Pluvio: Assembly Clone Search for Out-of-domain Architectures and Libraries through Transfer Learning and Conditional Variational Information Bottleneck
The practice of code reuse is crucial in software development for a faster
and more efficient development lifecycle. In reality, however, code reuse
practices lack proper control, resulting in issues such as vulnerability
propagation and intellectual property infringements. Assembly clone search, a
critical shift-right defence mechanism, has been effective in identifying
vulnerable code resulting from reuse in released executables. Recent studies on
assembly clone search demonstrate a trend towards using machine learning-based
methods to match assembly code variants produced by different toolchains.
However, these methods are limited to what they learn from a small number of
toolchain variants used in training, rendering them inapplicable to unseen
architectures and their corresponding compilation toolchain variants.
This paper presents the first study on the problem of assembly clone search
with unseen architectures and libraries. We propose incorporating human common
knowledge through large-scale pre-trained natural language models, in the form
of transfer learning, into current learning-based approaches for assembly clone
search. Transfer learning can aid in addressing the limitations of the existing
approaches, as it can bring in broader knowledge from human experts in assembly
code. We further address the sequence limit issue by proposing a reinforcement
learning agent to remove unnecessary and redundant tokens. Coupled with a new
Variational Information Bottleneck learning strategy, the proposed system
minimizes the reliance on potential indicators of architectures and
optimization settings, for a better generalization of unseen architectures. We
simulate the unseen architecture clone search scenarios and the experimental
results show the effectiveness of the proposed approach against the
state-of-the-art solutions.Comment: 13 pages and 4 figures. This work has been submitted to the IEEE for
possible publication. Copyright may be transferred without notice, after
which this version may no longer be accessibl
- …