QueryForm: A Simple Zero-shot Form Entity Query Framework

Zero-shot transfer learning for document understanding is a crucial yet under-investigated scenario to help reduce the high cost involved in annotating document entities. We present a novel query-based framework, QueryForm, that extracts entity values from form-like documents in a zero-shot fashion. QueryForm contains a dual prompting mechanism that composes both the document schema and a specific entity type into a query, which is used to prompt a Transformer model to perform a single entity extraction task. Furthermore, we propose to leverage large-scale query-entity pairs generated from form-like webpages with weak HTML annotations to pre-train QueryForm. By unifying pre-training and fine-tuning into the same query-based framework, QueryForm enables models to learn from structured documents containing various entities and layouts, leading to better generalization to target document types without the need for target-specific training data. QueryForm sets new state-of-the-art average F1 score on both the XFUND (+4.6%~10.1%) and the Payment (+3.2%~9.5%) zero-shot benchmark, with a smaller model size and no additional image input.Comment: Accepted to Findings of ACL 202

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

Table-based reasoning with large language models (LLMs) is a promising direction to tackle many table understanding tasks, such as table-based question answering and fact verification. Compared with generic reasoning, table-based reasoning requires the extraction of underlying semantics from both free-form questions and semi-structured tabular data. Chain-of-Thought and its similar approaches incorporate the reasoning chain in the form of textual context, but it is still an open question how to effectively leverage tabular data in the reasoning chain. We propose the Chain-of-Table framework, where tabular data is explicitly used in the reasoning chain as a proxy for intermediate thoughts. Specifically, we guide LLMs using in-context learning to iteratively generate operations and update the table to represent a tabular reasoning chain. LLMs can therefore dynamically plan the next operation based on the results of the previous ones. This continuous evolution of the table forms a chain, showing the reasoning process for a given tabular problem. The chain carries structured information of the intermediate results, enabling more accurate and reliable predictions. Chain-of-Table achieves new state-of-the-art performance on WikiTQ, FeTaQA, and TabFact benchmarks across multiple LLM choices.Comment: Accepted to ICLR 202

FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction

The recent advent of self-supervised pre-training techniques has led to a surge in the use of multimodal learning in form document understanding. However, existing approaches that extend the mask language modeling to other modalities require careful multi-task tuning, complex reconstruction target designs, or additional pre-training data. In FormNetV2, we introduce a centralized multimodal graph contrastive learning strategy to unify self-supervised pre-training for all modalities in one loss. The graph contrastive objective maximizes the agreement of multimodal representations, providing a natural interplay for all modalities without special customization. In addition, we extract image features within the bounding box that joins a pair of tokens connected by a graph edge, capturing more targeted visual cues without loading a sophisticated and separately pre-trained image embedder. FormNetV2 establishes new state-of-the-art performance on FUNSD, CORD, SROIE and Payment benchmarks with a more compact model size.Comment: Accepted to ACL 202

Emerging stability of forest productivity by mixing two species buffers temperature destabilizing effect

The increasing disturbances in monocultures around the world are testimony to their instability under global change. Many studies have claimed that temporal stability of productivity increases with species richness, although the ecological fundamentals have mainly been investigated through diversity experiments. To adequately manage forest ecosystems, it is necessary to have a comprehensive understanding of the effect of mixing species on the temporal stability of productivity and the way in which it is influenced by climate conditions across large geographical areas. Here, we used a unique dataset of 261 stands combining pure and two-species mixtures of four relevant tree species over a wide range of climate conditions in Europe to examine the effect of species mixing on the level and temporal stability of productivity. Structural equation modelling was employed to further explore the direct and indirect influence of climate, overyielding, species asynchrony and additive effect (i.e. temporal stability expected from the species growth in monospecific stands) on temporal stability in mixed forests. We showed that by adding only one tree species to monocultures, the level (overyielding: +6%) and stability (temporal stability: +12%) of stand growth increased significantly. We identified the key effect of temperature on destabilizing stand growth, which may be mitigated by mixing species. We further confirmed asynchrony as the main driver of temporal stability in mixed stands, through both the additive effect and species interactions, which modify between-species asynchrony in mixtures in comparison to monocultures. Synthesis and applications. This study highlights the emergent properties associated with mixing two species, which result in resource efficient and temporally stable production systems. We reveal the negative impact of mean temperature on temporal stability of forest productivity and how the stabilizing effect of mixing two species can counterbalance this impact. The overyielding and temporal stability of growth addressed in this paper are essential for ecosystem services closely linked with the level and rhythm of forest growth. Our results underline that mixing two species can be a realistic and effective nature-based climate solution, which could contribute towards meeting EU climate target policies