123,546 research outputs found
Textual Stylistic Variation: Choices, Genres and Individuals
This chapter argues for more informed target metrics for the statistical processing of stylistic variation in text collections. Much as operationalized relevance proved a useful goal to strive for in information retrieval, research in textual stylistics, whether application oriented or philologically inclined, needs goals formulated in terms of pertinence, relevance, and utility — notions that agree with reader ex- perience of text. Differences readers are aware of are mostly based on utility — not on textual characteristics per se. Mostly, readers report stylistic differences in terms of genres. Genres, while vague and undefined, are well-established and talked about: very early on, readers learn to distinguish genres. This chapter discusses variation given by genre, and contrasts it to variation occasioned by individual choice
Off-line vs. On-line Evaluation of Recommender Systems in Small E-commerce
In this paper, we present our work towards comparing on-line and off-line
evaluation metrics in the context of small e-commerce recommender systems.
Recommending on small e-commerce enterprises is rather challenging due to the
lower volume of interactions and low user loyalty, rarely extending beyond a
single session. On the other hand, we usually have to deal with lower volumes
of objects, which are easier to discover by users through various
browsing/searching GUIs.
The main goal of this paper is to determine applicability of off-line
evaluation metrics in learning true usability of recommender systems (evaluated
on-line in A/B testing). In total 800 variants of recommending algorithms were
evaluated off-line w.r.t. 18 metrics covering rating-based, ranking-based,
novelty and diversity evaluation. The off-line results were afterwards compared
with on-line evaluation of 12 selected recommender variants and based on the
results, we tried to learn and utilize an off-line to on-line results
prediction model.
Off-line results shown a great variance in performance w.r.t. different
metrics with the Pareto front covering 68\% of the approaches. Furthermore, we
observed that on-line results are considerably affected by the novelty of
users. On-line metrics correlates positively with ranking-based metrics (AUC,
MRR, nDCG) for novice users, while too high values of diversity and novelty had
a negative impact on the on-line results for them. For users with more visited
items, however, the diversity became more important, while ranking-based
metrics relevance gradually decrease.Comment: Submitted to ACM Hypertext 2020 Conferenc
Watch and Learn: Semi-Supervised Learning of Object Detectors from Videos
We present a semi-supervised approach that localizes multiple unknown object
instances in long videos. We start with a handful of labeled boxes and
iteratively learn and label hundreds of thousands of object instances. We
propose criteria for reliable object detection and tracking for constraining
the semi-supervised learning process and minimizing semantic drift. Our
approach does not assume exhaustive labeling of each object instance in any
single frame, or any explicit annotation of negative data. Working in such a
generic setting allow us to tackle multiple object instances in video, many of
which are static. In contrast, existing approaches either do not consider
multiple object instances per video, or rely heavily on the motion of the
objects present. The experiments demonstrate the effectiveness of our approach
by evaluating the automatically labeled data on a variety of metrics like
quality, coverage (recall), diversity, and relevance to training an object
detector.Comment: To appear in CVPR 201
Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters
Research has shown that personality is a key driver to improve engagement and
user experience in conversational systems. Conversational agents should also
maintain a consistent persona to have an engaging conversation with a user.
However, text generation datasets are often crowd sourced and thereby have an
averaging effect where the style of the generation model is an average style of
all the crowd workers that have contributed to the dataset. While one can
collect persona-specific datasets for each task, it would be an expensive and
time consuming annotation effort. In this work, we propose a novel transfer
learning framework which updates only of model parameters to learn
style specific attributes for response generation. For the purpose of this
study, we tackle the problem of stylistic story ending generation using the ROC
stories Corpus. We learn style specific attributes from the
PERSONALITY-CAPTIONS dataset. Through extensive experiments and evaluation
metrics we show that our novel training procedure can improve the style
generation by 200 over Encoder-Decoder baselines while maintaining on-par
content relevance metrics wit
DiffuCOMET: Contextual Commonsense Knowledge Diffusion
Inferring contextually-relevant and diverse commonsense to understand
narratives remains challenging for knowledge models. In this work, we develop a
series of knowledge models, DiffuCOMET, that leverage diffusion to learn to
reconstruct the implicit semantic connections between narrative contexts and
relevant commonsense knowledge. Across multiple diffusion steps, our method
progressively refines a representation of commonsense facts that is anchored to
a narrative, producing contextually-relevant and diverse commonsense inferences
for an input context. To evaluate DiffuCOMET, we introduce new metrics for
commonsense inference that more closely measure knowledge diversity and
contextual relevance. Our results on two different benchmarks, ComFact and
WebNLG+, show that knowledge generated by DiffuCOMET achieves a better
trade-off between commonsense diversity, contextual relevance and alignment to
known gold references, compared to baseline knowledge models
Relevance-based Word Embedding
Learning a high-dimensional dense representation for vocabulary terms, also
known as a word embedding, has recently attracted much attention in natural
language processing and information retrieval tasks. The embedding vectors are
typically learned based on term proximity in a large corpus. This means that
the objective in well-known word embedding algorithms, e.g., word2vec, is to
accurately predict adjacent word(s) for a given word or context. However, this
objective is not necessarily equivalent to the goal of many information
retrieval (IR) tasks. The primary objective in various IR tasks is to capture
relevance instead of term proximity, syntactic, or even semantic similarity.
This is the motivation for developing unsupervised relevance-based word
embedding models that learn word representations based on query-document
relevance information. In this paper, we propose two learning models with
different objective functions; one learns a relevance distribution over the
vocabulary set for each query, and the other classifies each term as belonging
to the relevant or non-relevant class for each query. To train our models, we
used over six million unique queries and the top ranked documents retrieved in
response to each query, which are assumed to be relevant to the query. We
extrinsically evaluate our learned word representation models using two IR
tasks: query expansion and query classification. Both query expansion
experiments on four TREC collections and query classification experiments on
the KDD Cup 2005 dataset suggest that the relevance-based word embedding models
significantly outperform state-of-the-art proximity-based embedding models,
such as word2vec and GloVe.Comment: to appear in the proceedings of The 40th International ACM SIGIR
Conference on Research and Development in Information Retrieval (SIGIR '17
- …