14,722 research outputs found
TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents
Recently, automatically extracting information from visually rich documents
(e.g., tickets and resumes) has become a hot and vital research topic due to
its widespread commercial value. Most existing methods divide this task into
two subparts: the text reading part for obtaining the plain text from the
original document images and the information extraction part for extracting key
contents. These methods mainly focus on improving the second, while neglecting
that the two parts are highly correlated. This paper proposes a unified
end-to-end information extraction framework from visually rich documents, where
text reading and information extraction can reinforce each other via a
well-designed multi-modal context block. Specifically, the text reading part
provides multi-modal features like visual, textual and layout features. The
multi-modal context block is developed to fuse the generated multi-modal
features and even the prior knowledge from the pre-trained language model for
better semantic representation. The information extraction part is responsible
for generating key contents with the fused context features. The framework can
be trained in an end-to-end trainable manner, achieving global optimization.
What is more, we define and group visually rich documents into four categories
across two dimensions, the layout and text type. For each document category, we
provide or recommend the corresponding benchmarks, experimental settings and
strong baselines for remedying the problem that this research area lacks the
uniform evaluation standard. Extensive experiments on four kinds of benchmarks
(from fixed layout to variable layout, from full-structured text to
semi-unstructured text) are reported, demonstrating the proposed method's
effectiveness. Data, source code and models are available
MicroConceptBERT: concept-relation based document information extraction framework.
Extracting information from documents is a crucial task in natural language processing research. Existing information extraction methodologies often focus on specific domains, such as medicine, education or finance, and are limited by language constraints. However, more comprehensive approaches that transcend document types, languages, contexts, and structures would significantly advance the field proposed in recent research. This study addresses this challenge by introducing microConceptBERT: a concept-relations-based framework for document information extraction, which offers flexibility for various document processing tasks while accounting for hierarchical, semantic, and heuristic features. The proposed framework has been applied to a question-answering task on benchmark datasets: SQUAD 2.0 and DOCVQA. Notably, the F1 evaluation metric attains an outperforming 87.01 performance rate on the SQUAD 2.0 dataset compared to baseline models: BERT-base and BERT-large models
Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration
Building document-grounded dialogue systems have received growing interest as
documents convey a wealth of human knowledge and commonly exist in enterprises.
Wherein, how to comprehend and retrieve information from documents is a
challenging research problem. Previous work ignores the visual property of
documents and treats them as plain text, resulting in incomplete modality. In
this paper, we propose a Layout-aware document-level Information Extraction
dataset, LIE, to facilitate the study of extracting both structural and
semantic knowledge from visually rich documents (VRDs), so as to generate
accurate responses in dialogue systems. LIE contains 62k annotations of three
extraction tasks from 4,061 pages in product and official documents, becoming
the largest VRD-based information extraction dataset to the best of our
knowledge. We also develop benchmark methods that extend the token-based
language model to consider layout features like humans. Empirical results show
that layout is critical for VRD-based extraction, and system demonstration also
verifies that the extracted knowledge can help locate the answers that users
care about.Comment: Accepted to ACM Multimedia (MM) Industry Track 202
- …