2,107 research outputs found
Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension
This study considers the task of machine reading at scale (MRS) wherein,
given a question, a system first performs the information retrieval (IR) task
of finding relevant passages in a knowledge source and then carries out the
reading comprehension (RC) task of extracting an answer span from the passages.
Previous MRS studies, in which the IR component was trained without considering
answer spans, struggled to accurately find a small number of relevant passages
from a large set of passages. In this paper, we propose a simple and effective
approach that incorporates the IR and RC tasks by using supervised multi-task
learning in order that the IR component can be trained by considering answer
spans. Experimental results on the standard benchmark, answering SQuAD
questions using the full Wikipedia as the knowledge source, showed that our
model achieved state-of-the-art performance. Moreover, we thoroughly evaluated
the individual contributions of our model components with our new Japanese
dataset and SQuAD. The results showed significant improvements in the IR task
and provided a new perspective on IR for RC: it is effective to teach which
part of the passage answers the question rather than to give only a relevance
score to the whole passage.Comment: 10 pages, 6 figure. Accepted as a full paper at CIKM 201
Fine-Grained Retrieval of Sports Plays using Tree-Based Alignment of Trajectories
We propose a novel method for effective retrieval of multi-agent spatiotemporal tracking data. Retrieval of spatiotemporal tracking data offers several unique challenges compared to conventional text-based retrieval settings. Most notably, the data is fine-grained meaning that the specific location of agents is important in describing behavior. Additionally, the data often contains tracks of multiple agents (e.g., multiple players in a sports game), which generally leads to a permutational alignment problem when performing relevance estimation. Due to the frequent position swap of agents, it is difficult to maintain the correspondence of agents, and such issues make the pairwise comparison problematic for multi-agent spatiotemporal data. To address this issue, we propose a tree-based method to estimate the relevance between multi-agent spatiotemporal tracks. It uses a hierarchical structure to perform multi-agent data alignment and partitioning in a coarse-to-fine fashion. We validate our approach via user studies with domain experts. Our results show that our method boosts performance in retrieving similar sports plays -- especially in interactive situations where the user selects a subset of trajectories compared to current state-of-the-art methods
Metric Optimization and Mainstream Bias Mitigation in Recommender Systems
The first part of this thesis focuses on maximizing the overall
recommendation accuracy. This accuracy is usually evaluated with some
user-oriented metric tailored to the recommendation scenario, but because
recommendation is usually treated as a machine learning problem, recommendation
models are trained to maximize some other generic criteria that does not
necessarily align with the criteria ultimately captured by the user-oriented
evaluation metric. Recent research aims at bridging this gap between training
and evaluation via direct ranking optimization, but still assumes that the
metric used for evaluation should also be the metric used for training. We
challenge this assumption, mainly because some metrics are more informative
than others. Indeed, we show that models trained via the optimization of a loss
inspired by Rank-Biased Precision (RBP) tend to yield higher accuracy, even
when accuracy is measured with metrics other than RBP. However, the superiority
of this RBP-inspired loss stems from further benefiting users who are already
well-served, rather than helping those who are not.
This observation inspires the second part of this thesis, where our focus
turns to helping non-mainstream users. These are users who are difficult to
recommend to either because there is not enough data to model them, or because
they have niche taste and thus few similar users to look at when recommending
in a collaborative way. These differences in mainstreamness introduce a bias
reflected in an accuracy gap between users or user groups, which we try to
narrow.Comment: PhD Thesis defended on Nov 14, 202
- …