773 research outputs found
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence
Topic models extract meaningful groups of words from documents, allowing for
a better understanding of data. However, the solutions are often not coherent
enough, and thus harder to interpret. Coherence can be improved by adding more
contextual knowledge to the model. Recently, neural topic models have become
available, while BERT-based representations have further pushed the state of
the art of neural models in general. We combine pre-trained representations and
neural topic models. Pre-trained BERT sentence embeddings indeed support the
generation of more meaningful and coherent topics than either standard LDA or
existing neural topic models. Results on four datasets show that our approach
effectively increases topic coherence
What are Public Concerns about ChatGPT? A Novel Self-Supervised Neural Topic Model Tells You
The recently released artificial intelligence conversational agent, ChatGPT,
has gained significant attention in academia and real life. A multitude of
early ChatGPT users eagerly explore its capabilities and share their opinions
on it via social media. Both user queries and social media posts express public
concerns regarding this advanced dialogue system. To mine public concerns about
ChatGPT, a novel Self-Supervised neural Topic Model (SSTM), which formalizes
topic modeling as a representation learning procedure, is proposed in this
paper. Extensive experiments have been conducted on Twitter posts about ChatGPT
and queries asked by ChatGPT users. And experimental results demonstrate that
the proposed approach could extract higher quality public concerns with
improved interpretability and diversity, surpassing the performance of
state-of-the-art approaches
A neural autoencoder approach for document ranking and query refinement in pharmacogenomic information retrieval
In this study, we investigate learning-to-
rank and query refinement approaches for
information retrieval in the pharmacogenomic domain. The goal is to improve the
information retrieval process of biomedical curators, who manually build knowledge bases for personalized medicine. We
study how to exploit the relationships be-
tween genes, variants, drugs, diseases and
outcomes as features for document ranking and query refinement.
For a supervised approach, we are faced with a
small amount of annotated data and a large
amount of unannotated data. Therefore,
we explore ways to use a neural document
auto-encoder in a semi-supervised approach. We show that a combination of established algorithms, feature-engineering
and a neural auto-encoder model yield
promising results in this setting
Tight Lower Bounds for Multiplicative Weights Algorithmic Families
We study the fundamental problem of prediction with expert advice and develop
regret lower bounds for a large family of algorithms for this problem. We
develop simple adversarial primitives, that lend themselves to various
combinations leading to sharp lower bounds for many algorithmic families. We
use these primitives to show that the classic Multiplicative Weights Algorithm
(MWA) has a regret of , there by completely closing
the gap between upper and lower bounds. We further show a regret lower bound of
for a much more general family of
algorithms than MWA, where the learning rate can be arbitrarily varied over
time, or even picked from arbitrary distributions over time. We also use our
primitives to construct adversaries in the geometric horizon setting for MWA to
precisely characterize the regret at for the case
of experts and a lower bound of
for the case of arbitrary number of experts
Active classification with comparison queries
We study an extension of active learning in which the learning algorithm may
ask the annotator to compare the distances of two examples from the boundary of
their label-class. For example, in a recommendation system application (say for
restaurants), the annotator may be asked whether she liked or disliked a
specific restaurant (a label query); or which one of two restaurants did she
like more (a comparison query).
We focus on the class of half spaces, and show that under natural
assumptions, such as large margin or bounded bit-description of the input
examples, it is possible to reveal all the labels of a sample of size using
approximately queries. This implies an exponential improvement over
classical active learning, where only label queries are allowed. We complement
these results by showing that if any of these assumptions is removed then, in
the worst case, queries are required.
Our results follow from a new general framework of active learning with
additional queries. We identify a combinatorial dimension, called the
\emph{inference dimension}, that captures the query complexity when each
additional query is determined by examples (such as comparison queries,
each of which is determined by the two compared examples). Our results for half
spaces follow by bounding the inference dimension in the cases discussed above.Comment: 23 pages (not including references), 1 figure. The new version
contains a minor fix in the proof of Lemma 4.
- …