9 research outputs found
Cross Temporal Recurrent Networks for Ranking Question Answer Pairs
Temporal gates play a significant role in modern recurrent-based neural
encoders, enabling fine-grained control over recursive compositional operations
over time. In recurrent models such as the long short-term memory (LSTM),
temporal gates control the amount of information retained or discarded over
time, not only playing an important role in influencing the learned
representations but also serving as a protection against vanishing gradients.
This paper explores the idea of learning temporal gates for sequence pairs
(question and answer), jointly influencing the learned representations in a
pairwise manner. In our approach, temporal gates are learned via 1D
convolutional layers and then subsequently cross applied across question and
answer for joint learning. Empirically, we show that this conceptually simple
sharing of temporal gates can lead to competitive performance across multiple
benchmarks. Intuitively, what our network achieves can be interpreted as
learning representations of question and answer pairs that are aware of what
each other is remembering or forgetting, i.e., pairwise temporal gating. Via
extensive experiments, we show that our proposed model achieves
state-of-the-art performance on two community-based QA datasets and competitive
performance on one factoid-based QA dataset.Comment: Accepted to AAAI201
Hashing based Answer Selection
Answer selection is an important subtask of question answering (QA), where
deep models usually achieve better performance. Most deep models adopt
question-answer interaction mechanisms, such as attention, to get vector
representations for answers. When these interaction based deep models are
deployed for online prediction, the representations of all answers need to be
recalculated for each question. This procedure is time-consuming for deep
models with complex encoders like BERT which usually have better accuracy than
simple encoders. One possible solution is to store the matrix representation
(encoder output) of each answer in memory to avoid recalculation. But this will
bring large memory cost. In this paper, we propose a novel method, called
hashing based answer selection (HAS), to tackle this problem. HAS adopts a
hashing strategy to learn a binary matrix representation for each answer, which
can dramatically reduce the memory cost for storing the matrix representations
of answers. Hence, HAS can adopt complex encoders like BERT in the model, but
the online prediction of HAS is still fast with a low memory cost. Experimental
results on three popular answer selection datasets show that HAS can outperform
existing models to achieve state-of-the-art performance
Learning to Truncate Ranked Lists for Information Retrieval
Ranked list truncation is of critical importance in a variety of professional
information retrieval applications such as patent search or legal search. The
goal is to dynamically determine the number of returned documents according to
some user-defined objectives, in order to reach a balance between the overall
utility of the results and user efforts. Existing methods formulate this task
as a sequential decision problem and take some pre-defined loss as a proxy
objective, which suffers from the limitation of local decision and non-direct
optimization. In this work, we propose a global decision based truncation model
named AttnCut, which directly optimizes user-defined objectives for the ranked
list truncation. Specifically, we take the successful transformer architecture
to capture the global dependency within the ranked list for truncation
decision, and employ the reward augmented maximum likelihood (RAML) for direct
optimization. We consider two types of user-defined objectives which are of
practical usage. One is the widely adopted metric such as F1 which acts as a
balanced objective, and the other is the best F1 under some minimal recall
constraint which represents a typical objective in professional search.
Empirical results over the Robust04 and MQ2007 datasets demonstrate the
effectiveness of our approach as compared with the state-of-the-art baselines
CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation
Dating and romantic relationships not only play a huge role in our personal
lives but also collectively influence and shape society. Today, many romantic
partnerships originate from the Internet, signifying the importance of
technology and the web in modern dating. In this paper, we present a text-based
computational approach for estimating the relationship compatibility of two
users on social media. Unlike many previous works that propose reciprocal
recommender systems for online dating websites, we devise a distant supervision
heuristic to obtain real world couples from social platforms such as Twitter.
Our approach, the CoupleNet is an end-to-end deep learning based estimator that
analyzes the social profiles of two users and subsequently performs a
similarity match between the users. Intuitively, our approach performs both
user profiling and match-making within a unified end-to-end framework.
CoupleNet utilizes hierarchical recurrent neural models for learning
representations of user profiles and subsequently coupled attention mechanisms
to fuse information aggregated from two users. To the best of our knowledge,
our approach is the first data-driven deep learning approach for our novel
relationship recommendation problem. We benchmark our CoupleNet against several
machine learning and deep learning baselines. Experimental results show that
our approach outperforms all approaches significantly in terms of precision.
Qualitative analysis shows that our model is capable of also producing
explainable results to users.Comment: Accepted at ICWSM 201