72 research outputs found
A Factoid Question Answering System for Vietnamese
In this paper, we describe the development of an end-to-end factoid question
answering system for the Vietnamese language. This system combines both
statistical models and ontology-based methods in a chain of processing modules
to provide high-quality mappings from natural language text to entities. We
present the challenges in the development of such an intelligent user interface
for an isolating language like Vietnamese and show that techniques developed
for inflectional languages cannot be applied "as is". Our question answering
system can answer a wide range of general knowledge questions with promising
accuracy on a test set.Comment: In the proceedings of the HQA'18 workshop, The Web Conference
Companion, Lyon, Franc
NeCo@ALQAC 2023: Legal Domain Knowledge Acquisition for Low-Resource Languages through Data Enrichment
In recent years, natural language processing has gained significant
popularity in various sectors, including the legal domain. This paper presents
NeCo Team's solutions to the Vietnamese text processing tasks provided in the
Automated Legal Question Answering Competition 2023 (ALQAC 2023), focusing on
legal domain knowledge acquisition for low-resource languages through data
enrichment. Our methods for the legal document retrieval task employ a
combination of similarity ranking and deep learning models, while for the
second task, which requires extracting an answer from a relevant legal article
in response to a question, we propose a range of adaptive techniques to handle
different question types. Our approaches achieve outstanding results on both
tasks of the competition, demonstrating the potential benefits and
effectiveness of question answering systems in the legal field, particularly
for low-resource languages.Comment: ISAILD@KSE 202
Exploring the State of the Art in Legal QA Systems
Answering questions related to the legal domain is a complex task, primarily
due to the intricate nature and diverse range of legal document systems.
Providing an accurate answer to a legal query typically necessitates
specialized knowledge in the relevant domain, which makes this task all the
more challenging, even for human experts. QA (Question answering systems) are
designed to generate answers to questions asked in human languages. They use
natural language processing to understand questions and search through
information to find relevant answers. QA has various practical applications,
including customer service, education, research, and cross-lingual
communication. However, they face challenges such as improving natural language
understanding and handling complex and ambiguous questions. Answering questions
related to the legal domain is a complex task, primarily due to the intricate
nature and diverse range of legal document systems. Providing an accurate
answer to a legal query typically necessitates specialized knowledge in the
relevant domain, which makes this task all the more challenging, even for human
experts. At this time, there is a lack of surveys that discuss legal question
answering. To address this problem, we provide a comprehensive survey that
reviews 14 benchmark datasets for question-answering in the legal field as well
as presents a comprehensive review of the state-of-the-art Legal Question
Answering deep learning models. We cover the different architectures and
techniques used in these studies and the performance and limitations of these
models. Moreover, we have established a public GitHub repository where we
regularly upload the most recent articles, open data, and source code. The
repository is available at:
\url{https://github.com/abdoelsayed2016/Legal-Question-Answering-Review}
A Survey of Pre-trained Language Models for Processing Scientific Text
The number of Language Models (LMs) dedicated to processing scientific text
is on the rise. Keeping pace with the rapid growth of scientific LMs (SciLMs)
has become a daunting task for researchers. To date, no comprehensive surveys
on SciLMs have been undertaken, leaving this issue unaddressed. Given the
constant stream of new SciLMs, appraising the state-of-the-art and how they
compare to each other remain largely unknown. This work fills that gap and
provides a comprehensive review of SciLMs, including an extensive analysis of
their effectiveness across different domains, tasks and datasets, and a
discussion on the challenges that lie ahead.Comment: Resources are available at https://github.com/Alab-NII/Awesome-SciL
English Machine Reading Comprehension Datasets: A Survey
This paper surveys 60 English Machine Reading Comprehension datasets, with a
view to providing a convenient resource for other researchers interested in
this problem. We categorize the datasets according to their question and answer
form and compare them across various dimensions including size, vocabulary,
data source, method of creation, human performance level, and first question
word. Our analysis reveals that Wikipedia is by far the most common data source
and that there is a relative lack of why, when, and where questions across
datasets.Comment: Will appear at EMNLP 2021. Dataset survey paper: 9 pages, 5 figures,
2 tables + attachmen
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
Coupled with the availability of large scale datasets, deep learning
architectures have enabled rapid progress on the Question Answering task.
However, most of those datasets are in English, and the performances of
state-of-the-art multilingual models are significantly lower when evaluated on
non-English data. Due to high data collection costs, it is not realistic to
obtain annotated data for each language one desires to support.
We propose a method to improve the Cross-lingual Question Answering
performance without requiring additional annotated data, leveraging Question
Generation models to produce synthetic samples in a cross-lingual fashion. We
show that the proposed method allows to significantly outperform the baselines
trained on English data only. We report a new state-of-the-art on four
multilingual datasets: MLQA, XQuAD, SQuAD-it and PIAF (fr).Comment: 7 page
GAIA: a benchmark for General AI Assistants
We introduce GAIA, a benchmark for General AI Assistants that, if solved,
would represent a milestone in AI research. GAIA proposes real-world questions
that require a set of fundamental abilities such as reasoning, multi-modality
handling, web browsing, and generally tool-use proficiency. GAIA questions are
conceptually simple for humans yet challenging for most advanced AIs: we show
that human respondents obtain 92\% vs. 15\% for GPT-4 equipped with plugins.
This notable performance disparity contrasts with the recent trend of LLMs
outperforming humans on tasks requiring professional skills in e.g. law or
chemistry. GAIA's philosophy departs from the current trend in AI benchmarks
suggesting to target tasks that are ever more difficult for humans. We posit
that the advent of Artificial General Intelligence (AGI) hinges on a system's
capability to exhibit similar robustness as the average human does on such
questions. Using GAIA's methodology, we devise 466 questions and their answer.
We release our questions while retaining answers to 300 of them to power a
leader-board available at https://huggingface.co/gaia-benchmark
- …