625 research outputs found
Training Curricula for Open Domain Answer Re-Ranking
In precision-oriented tasks like answer ranking, it is more important to rank
many relevant answers highly than to retrieve all relevant answers. It follows
that a good ranking strategy would be to learn how to identify the easiest
correct answers first (i.e., assign a high ranking score to answers that have
characteristics that usually indicate relevance, and a low ranking score to
those with characteristics that do not), before incorporating more complex
logic to handle difficult cases (e.g., semantic matching or reasoning). In this
work, we apply this idea to the training of neural answer rankers using
curriculum learning. We propose several heuristics to estimate the difficulty
of a given training sample. We show that the proposed heuristics can be used to
build a training curriculum that down-weights difficult samples early in the
training process. As the training process progresses, our approach gradually
shifts to weighting all samples equally, regardless of difficulty. We present a
comprehensive evaluation of our proposed idea on three answer ranking datasets.
Results show that our approach leads to superior performance of two leading
neural ranking architectures, namely BERT and ConvKNRM, using both pointwise
and pairwise losses. When applied to a BERT-based ranker, our method yields up
to a 4% improvement in MRR and a 9% improvement in P@1 (compared to the model
trained without a curriculum). This results in models that can achieve
comparable performance to more expensive state-of-the-art techniques.Comment: Accepted at SIGIR 2020 (long
Dependency-based Convolutional Neural Networks for Sentence Embedding
In sentence modeling and classification, convolutional neural network
approaches have recently achieved state-of-the-art results, but all such
efforts process word vectors sequentially and neglect long-distance
dependencies. To exploit both deep learning and linguistic structures, we
propose a tree-based convolutional neural network model which exploit various
long-distance relationships between words. Our model improves the sequential
baselines on all three sentiment and question classification tasks, and
achieves the highest published accuracy on TREC.Comment: this paper has been accepted by ACL 201
Combining information seeking services into a meta supply chain of facts
The World Wide Web has become a vital supplier of information that allows organizations to carry on such tasks as business intelligence, security monitoring, and risk assessments. Having a quick and reliable supply of correct facts from perspective is often mission critical. By following design science guidelines, we have explored ways to recombine facts from multiple sources, each with possibly different levels of responsiveness and accuracy, into one robust supply chain. Inspired by prior research on keyword-based meta-search engines (e.g., metacrawler.com), we have adapted the existing question answering algorithms for the task of analysis and triangulation of facts. We present a first prototype for a meta approach to fact seeking. Our meta engine sends a user's question to several fact seeking services that are publicly available on the Web (e.g., ask.com, brainboost.com, answerbus.com, NSIR, etc.) and analyzes the returned results jointly to identify and present to the user those that are most likely to be factually correct. The results of our evaluation on the standard test sets widely used in prior research support the evidence for the following: 1) the value-added of the meta approach: its performance surpasses the performance of each supplier, 2) the importance of using fact seeking services as suppliers to the meta engine rather than keyword driven search portals, and 3) the resilience of the meta approach: eliminating a single service does not noticeably impact the overall performance. We show that these properties make the meta-approach a more reliable supplier of facts than any of the currently available stand-alone services
Concept-based Interactive Query Expansion Support Tool (CIQUEST)
This report describes a three-year project (2000-03) undertaken in the Information Studies
Department at The University of Sheffield and funded by Resource, The Council for
Museums, Archives and Libraries. The overall aim of the research was to provide user
support for query formulation and reformulation in searching large-scale textual resources
including those of the World Wide Web. More specifically the objectives were: to investigate
and evaluate methods for the automatic generation and organisation of concepts derived from
retrieved document sets, based on statistical methods for term weighting; and to conduct
user-based evaluations on the understanding, presentation and retrieval effectiveness of
concept structures in selecting candidate terms for interactive query expansion.
The TREC test collection formed the basis for the seven evaluative experiments conducted in
the course of the project. These formed four distinct phases in the project plan. In the first
phase, a series of experiments was conducted to investigate further techniques for concept
derivation and hierarchical organisation and structure. The second phase was concerned with
user-based validation of the concept structures. Results of phases 1 and 2 informed on the
design of the test system and the user interface was developed in phase 3. The final phase
entailed a user-based summative evaluation of the CiQuest system.
The main findings demonstrate that concept hierarchies can effectively be generated from
sets of retrieved documents and displayed to searchers in a meaningful way. The approach
provides the searcher with an overview of the contents of the retrieved documents, which in
turn facilitates the viewing of documents and selection of the most relevant ones. Concept
hierarchies are a good source of terms for query expansion and can improve precision. The
extraction of descriptive phrases as an alternative source of terms was also effective. With
respect to presentation, cascading menus were easy to browse for selecting terms and for
viewing documents. In conclusion the project dissemination programme and future work are
outlined
Interaction history based answer formulation for question answering
With the rapid growth in information access methodologies, question answering has drawn considerable attention among others. Though question answering has emerged as an interesting new research domain, still it is vastly concentrated on question processing and answer extraction approaches. Latter steps like answer ranking, formulation and presentations are not treated in depth. Weakness we found in this arena is that answers that a particular user has acquired are not considered, when processing new questions. As a result, current systems are not capable of linking two questions such as βWhen is the Apple founded?β with a previously processed question βWhen is the Microsoft founded?β generating an answer in the form of βApple is founded one year later Microsoft founded, in 1976β. In this paper we present an approach towards question answering to devise an answer based on the questions already processed by the system for a particular user which is termed as interaction history for the user. Our approach is a combination of question processing, relation extraction and knowledge representation with inference models. During the process we primarily focus on acquiring knowledge and building up a scalable user model to formulate future answers based on current answers that same user has processed. According to evaluation we carried out based on the TREC resources shows that proposed technology is promising and effective in question answering
The Web as a Resource for Question Answering: Perspectives and Challenges
The vast amounts of information readily available on the World Wide Web can be effectively used for question answering in two fundamentally different ways. In the federated approach, techniques for handling semistructured data are applied to access Web sources as if they were databases, allowing large classes of common questions to be answered uniformly. In the distributed approach, largescale text-processing techniques are used to extract answers directly from unstructured Web documents. Because the Web is orders of magnitude larger than any human-collected corpus, question answering systems can capitalize on its unparalleled-levels of data redundancy. Analysis of real-world user questions reveals that the federated and distributed approaches complement each other nicely, suggesting a hybrid approach in future question answering systems
μ§μμλ΅ μμ€ν μ μν ν μ€νΈ λνΉ μ¬μΈ΅ μ κ²½λ§
νμλ
Όλ¬Έ (λ°μ¬) -- μμΈλνκ΅ λνμ : 곡과λν μ κΈ°Β·μ 보곡νλΆ, 2020. 8. μ κ΅λ―Ό.The question answering (QA) system has attracted huge interests due to its applicability in real-world applications. This dissertation proposes novel ranking algorithms for the QA system based on deep neural networks. We first tackle the long-text QA that requires the model to understand the excessively large sequence of text inputs. To solve this problem, we propose a hierarchical recurrent dual encoder that encodes texts from word-level to paragraph-level. We further propose a latent topic clustering method that utilizes semantic information in the target corpus, and thus it increases the performance of the QA system. Secondly, we investigate the short-text QA, where the information in text pairs are limited. To overcome the insufficiency, we combine a pretrained language model and an enhanced latent clustering method to the QA model. This novel architecture enables the model to utilizes additional information, resulting in achieving state-of-the-art performance for the standard answer-selection tasks (i.e., WikiQA, TREC-QA). Finally, we investigate detecting supporting sentences for complex QA system. As opposed to the previous studies, the model needs to understand the relationship between sentences to answer the question. Inspired by the hierarchical nature of the text, we propose a graph neural network-based model that iteratively propagates necessary information between text nodes and achieve the best performance among existing methods.λ³Έ νμ λ
Όλ¬Έμ λ₯ λ΄λ΄ λ€νΈμν¬ κΈ°λ° μ§μμλ΅ μμ€ν
μ κ΄ν λͺ¨λΈμ μ μνλ€. λ¨Όμ κΈ΄ λ¬Έμ₯μ λν μ§μμλ΅μ νκΈ° μν΄μ κ³μΈ΅ ꡬ쑰μ μ¬κ·μ κ²½λ§ λͺ¨λΈμ μ μνμλ€. μ΄λ₯Ό ν΅ν΄ λͺ¨λΈμ΄ μ£Όμ΄μ§ λ¬Έμ₯μ 짧μ μνμ€ λ¨μλ‘ ν¨μ¨μ μΌλ‘ λ€λ£° μ μκ² νμ¬ ν° μ±λ₯ ν₯μμ μ»μλ€. λν νμ΅ κ³Όμ μμ λ°μ΄ν° μμ λ΄ν¬λ ν ν½μ μλ λΆλ₯νλ λͺ¨λΈμ μ μνκ³ , μ΄λ₯Ό κΈ°μ‘΄ μ§μμλ΅ λͺ¨λΈμ λ³ν©νμ¬ μΆκ° μ±λ₯ κ°μ μ μ΄λ£¨μλ€. μ΄μ΄μ§λ μ°κ΅¬λ‘ 짧μ λ¬Έμ₯μ λν μ§μμλ΅ λͺ¨λΈμ μ μνμλ€. λ¬Έμ₯μ κΈΈμ΄κ° 짧μμ§μλ‘ λ¬Έμ₯ μμμ μ»μ μ μλ μ 보μ μλ μ€μ΄λ€κ² λλ€. μ°λ¦¬λ μ΄λ¬ν λ¬Έμ λ₯Ό ν΄κ²°νκΈ° μν΄, μ¬μ νμ΅λ μΈμ΄ λͺ¨λΈκ³Ό μλ‘μ΄ ν ν½ ν΄λ¬μ€ν°λ§ κΈ°λ²μ μ μ©νμλ€. μ μν λͺ¨λΈμ μ’
λ 짧μ λ¬Έμ₯ μ§μμλ΅ μ°κ΅¬ μ€ κ°μ₯ μ’μ μ±λ₯μ νλνμλ€. λ§μ§λ§μΌλ‘ μ¬λ¬ λ¬Έμ₯ μ¬μ΄μ κ΄κ³λ₯Ό μ΄μ©νμ¬ λ΅λ³μ μ°ΎμμΌ νλ μ§μμλ΅ μ°κ΅¬λ₯Ό μ§ννμλ€. μ°λ¦¬λ λ¬Έμ λ΄ κ° λ¬Έμ₯μ κ·Έλνλ‘ λμνν ν μ΄λ₯Ό νμ΅ν μ μλ κ·Έλν λ΄λ΄ λ€νΈμν¬λ₯Ό μ μνμλ€. μ μν λͺ¨λΈμ κ° λ¬Έμ₯μ κ΄κ³μ±μ μ±κ³΅μ μΌλ‘ κ³μ°νμκ³ , μ΄λ₯Ό ν΅ν΄ 볡μ‘λκ° λμ μ§μμλ΅ μμ€ν
μμ κΈ°μ‘΄μ μ μλ λͺ¨λΈλ€κ³Ό λΉκ΅νμ¬ κ°μ₯ μ’μ μ±λ₯μ νλνμλ€.1 Introduction 1
2 Background 8
2.1 Textual Data Representation 8
2.2 Encoding Sequential Information in Text 12
3 Question-Answer Pair Ranking for Long Text 16
3.1 Related Work 18
3.2 Method 19
3.2.1 Baseline Approach 19
3.2.2 Proposed Approaches (HRDE+LTC) 22
3.3 Experimental Setup and Dataset 26
3.3.1 Dataset 26
3.3.2 Consumer Product Question Answering Corpus 30
3.3.3 Implementation Details 32
3.4 Empirical Results 34
3.4.1 Comparison with other methods 35
3.4.2 Degradation Comparison for Longer Texts 37
3.4.3 Effects of the LTC Numbers 38
3.4.4 Comprehensive Analysis of LTC 38
3.5 Further Investigation on Ranking Lengthy Document 40
3.5.1 Problem and Dataset 41
3.5.2 Methods 45
3.5.3 Experimental Results 51
3.6 Conclusion 55
4 Answer-Selection for Short Sentence 56
4.1 Related Work 57
4.2 Method 59
4.2.1 Baseline approach 59
4.2.2 Proposed Approaches (Comp-Clip+LM+LC+TL) 62
4.3 Experimental Setup and Dataset 66
4.3.1 Dataset 66
4.3.2 Implementation Details 68
4.4 Empirical Results 69
4.4.1 Comparison with Other Methods 69
4.4.2 Impact of Latent Clustering 72
4.5 Conclusion 72
5 Supporting Sentence Detection for Question Answering 73
5.1 Related Work 75
5.2 Method 76
5.2.1 Baseline approaches 76
5.2.2 Proposed Approach (Propagate-Selector) 78
5.3 Experimental Setup and Dataset 82
5.3.1 Dataset 82
5.3.2 Implementation Details 83
5.4 Empirical Results 85
5.4.1 Comparisons with Other Methods 85
5.4.2 Hop Analysis 86
5.4.3 Impact of Various Graph Topologies 88
5.4.4 Impact of Node Representation 91
5.5 Discussion 92
5.6 Conclusion 93
6 Conclusion 94Docto
- β¦