Question answering on web data : the QA evaluation in Quaero

International audienceIn the QA and information retrieval domains, progress has been assessed via evaluation campaigns(Clef, Ntcir, Equer, Trec). In these evaluations, the systems handle independent questions and should provide one answer to each question, extracted from textual data, for both open domain and restricted domain. Quæro is a program promoting research and industrial innovation on technologies for automatic analysis and classification of multimedia and multilingual documents. Among the many research areas concerned by Quæro. The Quaero project organized a series of evaluations of Question Answering on Web Data systems in 2008 and 2009. For each language, English and French the full corpus has a size of around 20Gb for 2.5M documents. We describe the task and corpora, and especially the methodologies used in 2008 to construct the test of question and a new one in the 2009 campaign. Six types of questions were addressed, factual, Non-factual(How, Why, What), List, Boolean. A description of the participating systems and the obtained results is provided. We show the difficulty for a question-answering system to work with complex data and questions

Encyclopedia Expert

Předmětem projektu je systém pro zodpovídání otázek formulovaných v přirozeném jazyce. Práce pojednává nejprve o problémech spjatých se systému tohoto druhu a o některých uplatňovaných přístupech. Důraz je kladen na povrchové metody, které nejsou tolik náročné na dostupnost lingvistických zdrojů. V praktické části je pak popsán návrh systému, který zodpovídá faktografické otázky s využitím české Wikipedie jako zdroje informací. Extrakce odpovědí je založena zčásti na specifických rysech Wikipedie a zčásti na ručně předdefinovaných vzorech. Výsledky ukazují, že pro zodpovídání jednoduchých otázek je systém výrazně přínosnější než běžný vyhledávací stroj.This project focuses on a system that answers questions formulated in natural language. Firstly, the report discusses problems associated with question answering systems and some commonly employed approaches. Emphasis is laid on shallow methods, which do not require many linguistic resources. The second part describes our work on a system that answers factoid questions, utilizing Czech Wikipedia as a source of information. Answer extraction is partly based on specific features of Wikipedia and partly on pre-defined patterns. Results show that for answering simple questions, the system provides significant improvements in comparison with a standard search engine.

A study of the use of natural language processing for conversational agents

Language is a mark of humanity and conscience, with the conversation (or dialogue) as one of the most fundamental manners of communication that we learn as children. Therefore one way to make a computer more attractive for interaction with users is through the use of natural language. Among the systems with some degree of language capabilities developed, the Eliza chatterbot is probably the first with a focus on dialogue. In order to make the interaction more interesting and useful to the user there are other approaches besides chatterbots, like conversational agents. These agents generally have, to some degree, properties like: a body (with cognitive states, including beliefs, desires and intentions or objectives); an interactive incorporation in the real or virtual world (including perception of events, communication, ability to manipulate the world and communicate with others); and behavior similar to a human (including affective abilities). This type of agents has been called by several terms, including animated agents or embedded conversational agents (ECA). A dialogue system has six basic components. (1) The speech recognition component is responsible for translating the user’s speech into text. (2) The Natural Language Understanding component produces a semantic representation suitable for dialogues, usually using grammars and ontologies. (3) The Task Manager chooses the concepts to be expressed to the user. (4) The Natural Language Generation component defines how to express these concepts in words. (5) The dialog manager controls the structure of the dialogue. (6) The synthesizer is responsible for translating the agents answer into speech. However, there is no consensus about the necessary resources for developing conversational agents and the difficulties involved (especially in resource-poor languages). This work focuses on the influence of natural language components (dialogue understander and manager) and analyses, in particular the use of parsing systems as part of developing conversational agents with more flexible language capabilities. This work analyses what kind of parsing resources contributes to conversational agents and discusses how to develop them targeting Portuguese, which is a resource-poor language. To do so we analyze approaches to the understanding of natural language, and identify parsing approaches that offer good performance, based on which we develop a prototype to evaluate the impact of using a parser in a conversational agent.linguagem é uma marca da humanidade e da consciência, sendo a conversação (ou diálogo) uma das maneiras de comunicacão mais fundamentais que aprendemos quando crianças. Por isso uma forma de fazer um computador mais atrativo para interação com usuários é usando linguagem natural. Dos sistemas com algum grau de capacidade de linguagem desenvolvidos, o chatterbot Eliza é, provavelmente, o primeiro sistema com foco em diálogo. Com o objetivo de tornar a interação mais interessante e útil para o usuário há outras aplicações alem de chatterbots, como agentes conversacionais. Estes agentes geralmente possuem, em algum grau, propriedades como: corpo (com estados cognitivos, incluindo crenças, desejos e intenções ou objetivos); incorporação interativa no mundo real ou virtual (incluindo percepções de eventos, comunicação, habilidade de manipular o mundo e comunicar com outros agentes); e comportamento similar ao humano (incluindo habilidades afetivas). Este tipo de agente tem sido chamado de diversos nomes como agentes animados ou agentes conversacionais incorporados. Um sistema de diálogo possui seis componentes básicos. (1) O componente de reconhecimento de fala que é responsável por traduzir a fala do usuário em texto. (2) O componente de entendimento de linguagem natural que produz uma representação semântica adequada para diálogos, normalmente utilizando gramáticas e ontologias. (3) O gerenciador de tarefa que escolhe os conceitos a serem expressos ao usuário. (4) O componente de geração de linguagem natural que define como expressar estes conceitos em palavras. (5) O gerenciador de diálogo controla a estrutura do diálogo. (6) O sintetizador de voz é responsável por traduzir a resposta do agente em fala. No entanto, não há consenso sobre os recursos necessários para desenvolver agentes conversacionais e a dificuldade envolvida nisso (especialmente em línguas com poucos recursos disponíveis). Este trabalho foca na influência dos componentes de linguagem natural (entendimento e gerência de diálogo) e analisa em especial o uso de sistemas de análise sintática (parser) como parte do desenvolvimento de agentes conversacionais com habilidades de linguagem mais flexível. Este trabalho analisa quais os recursos do analisador sintático contribuem para agentes conversacionais e aborda como os desenvolver, tendo como língua alvo o português (uma língua com poucos recursos disponíveis). Para isto, analisamos as abordagens de entendimento de linguagem natural e identificamos as abordagens de análise sintática que oferecem um bom desempenho. Baseados nesta análise, desenvolvemos um protótipo para avaliar o impacto do uso de analisador sintático em um agente conversacional

RANCANG BANGUN APLIKASI QUESTION ANSWERING (QA) SYSTEM PADA TERJEMAHAN AL QURAN MENGGUNAKAN EPHYRA FRAMEWORK

Al Quran merupakan tuntunan yang wajib diikuti oleh umat Islam. Permasalahan-permasalahan agama dalam kehidupan sehari-hari banyak termaktub pada Al Quran. Dalam Al Quran suatu permasalahan tidak hanya mengacu pada satu ayat ataupun satu surat saja sehingga dibutuhkan waktu yang lama dalam proses pencarian secara manual mengingat banyaknya jumlah ayat dan surat yang terkandung dalam Al Quran. Untuk itu dibutuhkan sebuah aplikasi yang mudah dalam mengenali dan mencari masalah yang dibutuhkan oleh pengguna sehingga sistem tersebut dapat menampilkan ayat-ayat Al Quran sebagai referensi. Berdasarkan hal tersebut pada penelitian ini, dibangun aplikasi QA System pada terjemahan Al Quran menggunakan Ephyra Framewok dengan IR system yang digunakan adalah model ruang vektor yang dibangun dengan bahasa PHP. Dalam penelitian ini digunakan tiga kategori pertanyaan yang meliputi orang, tempat dan waktu dengan kata tanya yang digunakan adalah siapa, siapakah, kapan, kapankah, dimana, dimanakah, kemana, kemanakah, darimana, dan darimanakah. Hasilnya adalah secara keseluruhan aplikasi QA System ini memiliki nilai presisi sebesar 42,31% . Kata kunci: Ephyra Framework, Information Retrieval, Model Ruang Vektor, Question Answering System, Terjemahan Al Qura