An Unsupervised Model with Attention Autoencoders for Question Retrieval

Question retrieval is a crucial subtask for community question answering. Previous research focus on supervised models which depend heavily on training data and manual feature engineering. In this paper, we propose a novel unsupervised framework, namely reduced attentive matching network (RAMN), to compute semantic matching between two questions. Our RAMN integrates together the deep semantic representations, the shallow lexical mismatching information and the initial rank produced by an external search engine. For the first time, we propose attention autoencoders to generate semantic representations of questions. In addition, we employ lexical mismatching to capture surface matching between two questions, which is derived from the importance of each word in a question. We conduct experiments on the open CQA datasets of SemEval-2016 and SemEval-2017. The experimental results show that our unsupervised model obtains comparable performance with the state-of-the-art supervised methods in SemEval-2016 Task 3, and outperforms the best system in SemEval-2017 Task 3 by a wide margin

Autoencoding the Retrieval Relevance of Medical Images

Content-based image retrieval (CBIR) of medical images is a crucial task that can contribute to a more reliable diagnosis if applied to big data. Recent advances in feature extraction and classification have enormously improved CBIR results for digital images. However, considering the increasing accessibility of big data in medical imaging, we are still in need of reducing both memory requirements and computational expenses of image retrieval systems. This work proposes to exclude the features of image blocks that exhibit a low encoding error when learned by a $n/p/n$ autoencoder ($p\!<\!n$). We examine the histogram of autoendcoding errors of image blocks for each image class to facilitate the decision which image regions, or roughly what percentage of an image perhaps, shall be declared relevant for the retrieval task. This leads to reduction of feature dimensionality and speeds up the retrieval process. To validate the proposed scheme, we employ local binary patterns (LBP) and support vector machines (SVM) which are both well-established approaches in CBIR research community. As well, we use IRMA dataset with 14,410 x-ray images as test data. The results show that the dimensionality of annotated feature vectors can be reduced by up to 50% resulting in speedups greater than 27% at expense of less than 1% decrease in the accuracy of retrieval when validating the precision and recall of the top 20 hits.Comment: To appear in proceedings of The 5th International Conference on Image Processing Theory, Tools and Applications (IPTA'15), Nov 10-13, 2015, Orleans, Franc

Semi-Supervised Learning for Neural Machine Translation

While end-to-end neural machine translation (NMT) has made remarkable progress recently, NMT systems only rely on parallel corpora for parameter estimation. Since parallel corpora are usually limited in quantity, quality, and coverage, especially for low-resource languages, it is appealing to exploit monolingual corpora to improve NMT. We propose a semi-supervised approach for training NMT models on the concatenation of labeled (parallel corpora) and unlabeled (monolingual corpora) data. The central idea is to reconstruct the monolingual corpora using an autoencoder, in which the source-to-target and target-to-source translation models serve as the encoder and decoder, respectively. Our approach can not only exploit the monolingual corpora of the target language, but also of the source language. Experiments on the Chinese-English dataset show that our approach achieves significant improvements over state-of-the-art SMT and NMT systems.Comment: Corrected a typ

Modelos densos e híbridos para recuperação de informação

As in the era of Big Data, there is the need of finding information in an easy and fast way, being imperative for a search system to understand more efficiently the user intent. Dense Retrieval focuses on this idea, by allowing the models to capture the underlying meaning of the queries and documents. Current models already surpass the classical BM-25 model in terms of accuracy. However, due to the use of a high number of dimensions to create representations of the queries and documents, the dense models are still not optimized in terms of their efficiency at a search level. This work focuses on evaluating the need for that high number of dimensions, by analyzing different dimensionality reduction methods, trained for different purposes, and comparing the trade-offs between efficiency and accuracy.Na era de Big Data em que nos encontramos, existe a necessidade de encontrar informação de uma forma mais fácil e mais rápida, sendo imperativo para um sistema de pesquisa entender eficientemente a intenção do utilizador. O campo de Dense Retrieval foca-se nesta ideia, permitindo que os modelos capturem os aspetos semânticos de queries e documentos. Modelos atuais já superam o modelo clássico BM-25 em termos de eficácia. No entanto, devido à aplicação de um número elevado de dimensões para criar as representações de queries e documentos, estes modelos densos ainda não estão otimizados em termos de desempenho ao nível da pesquisa. Este trabalho foca-se em avaliar a necessidade desse número elevado de dimensões, analisando diferentes métodos de redução de dimensionalidade, orientados para diferentes objectivos, e em comparar pontos de equilíbrio entre eficiência e precisão.Mestrado em Engenharia Informátic