5 research outputs found
Cross-Language Question Re-Ranking
We study how to find relevant questions in community forums when the language
of the new questions is different from that of the existing questions in the
forum. In particular, we explore the Arabic-English language pair. We compare a
kernel-based system with a feed-forward neural network in a scenario where a
large parallel corpus is available for training a machine translation system,
bilingual dictionaries, and cross-language word embeddings. We observe that
both approaches degrade the performance of the system when working on the
translated text, especially the kernel-based system, which depends heavily on a
syntactic kernel. We address this issue using a cross-language tree kernel,
which compares the original Arabic tree to the English trees of the related
questions. We show that this kernel almost closes the performance gap with
respect to the monolingual system. On the neural network side, we use the
parallel corpus to train cross-language embeddings, which we then use to
represent the Arabic input and the English related questions in the same space.
The results also improve to close to those of the monolingual neural network.
Overall, the kernel system shows a better performance compared to the neural
network in all cases.Comment: SIGIR-2017; Community Question Answering; Cross-language Approaches;
Question Retrieval; Kernel-based Methods; Neural Networks; Distributed
Representation
Deep learning in clinical natural language processing: a methodical review.
OBJECTIVE: This article methodically reviews the literature on deep learning (DL) for natural language processing (NLP) in the clinical domain, providing quantitative analysis to answer 3 research questions concerning methods, scope, and context of current research.
MATERIALS AND METHODS: We searched MEDLINE, EMBASE, Scopus, the Association for Computing Machinery Digital Library, and the Association for Computational Linguistics Anthology for articles using DL-based approaches to NLP problems in electronic health records. After screening 1,737 articles, we collected data on 25 variables across 212 papers.
RESULTS: DL in clinical NLP publications more than doubled each year, through 2018. Recurrent neural networks (60.8%) and word2vec embeddings (74.1%) were the most popular methods; the information extraction tasks of text classification, named entity recognition, and relation extraction were dominant (89.2%). However, there was a long tail of other methods and specific tasks. Most contributions were methodological variants or applications, but 20.8% were new methods of some kind. The earliest adopters were in the NLP community, but the medical informatics community was the most prolific.
DISCUSSION: Our analysis shows growing acceptance of deep learning as a baseline for NLP research, and of DL-based NLP in the medical community. A number of common associations were substantiated (eg, the preference of recurrent neural networks for sequence-labeling named entity recognition), while others were surprisingly nuanced (eg, the scarcity of French language clinical NLP with deep learning).
CONCLUSION: Deep learning has not yet fully penetrated clinical NLP and is growing rapidly. This review highlighted both the popular and unique trends in this active field
Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval
Dual-Encoders is a promising mechanism for answer retrieval in question
answering (QA) systems. Currently most conventional Dual-Encoders learn the
semantic representations of questions and answers merely through matching
score. Researchers proposed to introduce the QA interaction features in scoring
function but at the cost of low efficiency in inference stage. To keep
independent encoding of questions and answers during inference stage,
variational auto-encoder is further introduced to reconstruct answers
(questions) from question (answer) embeddings as an auxiliary task to enhance
QA interaction in representation learning in training stage. However, the needs
of text generation and answer retrieval are different, which leads to hardness
in training. In this work, we propose a framework to enhance the Dual-Encoders
model with question answer cross-embeddings and a novel Geometry Alignment
Mechanism (GAM) to align the geometry of embeddings from Dual-Encoders with
that from Cross-Encoders. Extensive experimental results show that our
framework significantly improves Dual-Encoders model and outperforms the
state-of-the-art method on multiple answer retrieval datasets.Comment: Findings of EMNLP 2021(10 pages