27,309 research outputs found
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning
Over the past years, distributed semantic representations have proved to be
effective and flexible keepers of prior knowledge to be integrated into
downstream applications. This survey focuses on the representation of meaning.
We start from the theoretical background behind word vector space models and
highlight one of their major limitations: the meaning conflation deficiency,
which arises from representing a word with all its possible meanings as a
single vector. Then, we explain how this deficiency can be addressed through a
transition from the word level to the more fine-grained level of word senses
(in its broader acceptation) as a method for modelling unambiguous lexical
meaning. We present a comprehensive overview of the wide range of techniques in
the two main branches of sense representation, i.e., unsupervised and
knowledge-based. Finally, this survey covers the main evaluation procedures and
applications for this type of representation, and provides an analysis of four
of its important aspects: interpretability, sense granularity, adaptability to
different domains and compositionality.Comment: 46 pages, 8 figures. Published in Journal of Artificial Intelligence
Researc
ADGym: Design Choices for Deep Anomaly Detection
Deep learning (DL) techniques have recently found success in anomaly
detection (AD) across various fields such as finance, medical services, and
cloud computing. However, most of the current research tends to view deep AD
algorithms as a whole, without dissecting the contributions of individual
design choices like loss functions and network architectures. This view tends
to diminish the value of preliminary steps like data preprocessing, as more
attention is given to newly designed loss functions, network architectures, and
learning paradigms. In this paper, we aim to bridge this gap by asking two key
questions: (i) Which design choices in deep AD methods are crucial for
detecting anomalies? (ii) How can we automatically select the optimal design
choices for a given AD dataset, instead of relying on generic, pre-existing
solutions? To address these questions, we introduce ADGym, a platform
specifically crafted for comprehensive evaluation and automatic selection of AD
design elements in deep methods. Our extensive experiments reveal that relying
solely on existing leading methods is not sufficient. In contrast, models
developed using ADGym significantly surpass current state-of-the-art
techniques.Comment: NeurIPS 2023. The first three authors contribute equally. Code
available at https://github.com/Minqi824/ADGy
- …