Islamic Applications of Automatic Question-Answering

A search engine aims to retrieve full documents whereas a question answering system aims to extract the exact answer. A question answering system involves the process of accepting a NL (Natural Language) question, analyzing, and processing to match against a knowledge base to generate the right answer from documents for users. For the Holy Quran this involves accepting the NL question and processing it to retrieve the right verse or verses from our Quran knowledge base. Question answering systems can use two types of algorithms: rule based techniques and/or AI (Artificial Intelligence) based techniques. Question Answering systems have three main components: question classification, information retrieval and answer extraction. We present a rule-based system for the Holy Quran that retrieves the right verse(s) from the Holy Quran instead of generating NL answers. We use a java program to extract the answer from a MS-Access database which contains our knowledge base for our Quran question answering system. We find that the system gives better results for the question after improving the system by removing stop words

Islamic Applications of Automatic Question-Answering

search engine aims to retrieve full documents whereas a question answering system aims to extract the exact answer. A question answering system involves the process of accepting a NL (Natural Language) question, analyzing, and processing to match against a knowledge base to generate the right answer from documents for users. For the Holy Quran this involves accepting the NL question and processing it to retrieve the right verse or verses from our Quran knowledge base. Question answering systems can use two types of algorithms: rule based techniques and/or AI (Artificial Intelligence) based techniques. Question Answering systems have three main components: question classification, information retrieval and answer extraction. We present a rule-based system for the Holy Quran that retrieves the right verse(s) from the Holy Quran instead of generating NL answers. We use a java program to extract the answer from a MS-Access database which contains our knowledge base for our Quran question answering system. We find that the system gives better results for the question after improving the system by removing stop words

How to Rank Answers in Text Mining

In this thesis, we mainly focus on case studies about answers. We present the methodology CEW-DTW and assess its performance about ranking quality. Based on the CEW-DTW, we improve this methodology by combining Kullback-Leibler divergence with CEW-DTW, since Kullback-Leibler divergence can check the difference of probability distributions in two sequences. However, CEW-DTW and KL-CEW-DTW do not care about the effect of noise and keywords from the viewpoint of probability distribution. Therefore, we develop a new methodology, the General Entropy, to see how probabilities of noise and keywords affect answer qualities. We firstly analyze some properties of the General Entropy, such as the value range of the General Entropy. Especially, we try to find an objective goal, which can be regarded as a standard to assess answers. Therefore, we introduce the maximum general entropy. We try to use the general entropy methodology to find an imaginary answer with the maximum entropy from the mathematical viewpoint (though this answer may not exist). This answer can also be regarded as an “ideal” answer. By comparing maximum entropy probabilities and global probabilities of noise and keywords respectively, the maximum entropy probability of noise is smaller than the global probability of noise, maximum entropy probabilities of chosen keywords are larger than global probabilities of keywords in some conditions. This allows us to determinably select the max number of keywords. We also use Amazon dataset and a small group of survey to assess the general entropy. Though these developed methodologies can analyze answer qualities, they do not incorporate the inner connections among keywords and noise. Based on the Markov transition matrix, we develop the Jump Probability Entropy. We still adapt Amazon dataset to compare maximum jump entropy probabilities and global jump probabilities of noise and keywords respectively. Finally, we give steps about how to get answers from Amazon dataset, including obtaining original answers from Amazon dataset, removing stopping words and collinearity. We compare our developed methodologies to see if these methodologies are consistent. Also, we introduce Wald–Wolfowitz runs test and compare it with developed methodologies to verify their relationships. Depending on results of comparison, we get conclusions about consistence of these methodologies and illustrate future plans

A Boolean based Question Answering System

The search engine searches the information according to the key words and provides users with related links, which need users to review and find the direct information among a large number of webpages. To avoid this drawback and improve the search results from search engine, we implemented a Boolean based Question Answering System. This system used Boolean Retrieval Model to analyze and match the text information from corresponding webpages in the document indexing step when users ask a Boolean expression based question. To evaluate system and analyze Boolean Retrieval Model, we used the data set from TREC (Text Retrieval Conference) to finish our experiment. Different Boolean operators in the questions such as AND, OR has been evaluated separately which is clear to analyze the effectiveness for each of them. We also evaluate the overall performance for this system

Self-Aware Learning system

This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019Cataloged from student-submitted PDF of thesis.Includes bibliographical references (pages 67-68).In this thesis, I take a step towards understanding how and why humans learn to solve problems about their solving of problems. I present a general-purpose neural reinforcement learning system called SAL, which can learn to think about its own problem solving, and use this capability to learn how to solve problems at another level. I show that SAL can use self-reference to articulate, and learn to articulate, its thoughts to a human, and internalize and apply a human's help, in natural language. I also demonstrate that SAL's abilities are enabled by an internal representation that shares important properties with, and is easily converted between, natural language. On the practical side, I argue that SAL can inform production question answering systems research. SAL can answer multi-step questions that are grounded in the world by extracting operational knowledge from pre-trained word embeddings. As an example, SAL knows how to use the action associated with \grab [the] diesel jug" to get closer to a solution, given the state of a physical world and a goal. And SAL can do this without any actual experience using (and without ever being told by a human about) any action associated with \grab" or the argument \diesel jug." SAL can do so with both very little training reward data and without assuming anything about the operational meaning of a particular lexical item, or composition of them, at first. Alternatively, typical neural reinforcement learning systems can not learn like SAL; they only work with a level of data that would be difficult to achieve in the real world. SAL's implementation, trained models, analysis code, and instructions, are at https://github.com/TristanThrush/sal. It is easy to add new problems (even in new domains) that you want SAL to learn.by Tristan Andrew Fraser Thrush.M. Eng.M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Scienc