3,095 research outputs found
Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning
Neural semantic parsing has achieved impressive results in recent years, yet
its success relies on the availability of large amounts of supervised data. Our
goal is to learn a neural semantic parser when only prior knowledge about a
limited number of simple rules is available, without access to either annotated
programs or execution results. Our approach is initialized by rules, and
improved in a back-translation paradigm using generated question-program pairs
from the semantic parser and the question generator. A phrase table with
frequent mapping patterns is automatically derived, also updated as training
progresses, to measure the quality of generated instances. We train the model
with model-agnostic meta-learning to guarantee the accuracy and stability on
examples covered by rules, and meanwhile acquire the versatility to generalize
well on examples uncovered by rules. Results on three benchmark datasets with
different domains and programs show that our approach incrementally improves
the accuracy. On WikiSQL, our best model is comparable to the SOTA system
learned from denotations
Semantic Parsing in Limited Resource Conditions
This thesis explores challenges in semantic parsing, specifically focusing on
scenarios with limited data and computational resources. It offers solutions
using techniques like automatic data curation, knowledge transfer, active
learning, and continual learning.
For tasks with no parallel training data, the thesis proposes generating
synthetic training examples from structured database schemas. When there is
abundant data in a source domain but limited parallel data in a target domain,
knowledge from the source is leveraged to improve parsing in the target domain.
For multilingual situations with limited data in the target languages, the
thesis introduces a method to adapt parsers using a limited human translation
budget. Active learning is applied to select source-language samples for manual
translation, maximizing parser performance in the target language. In addition,
an alternative method is also proposed to utilize machine translation services,
supplemented by human-translated data, to train a more effective parser.
When computational resources are limited, a continual learning approach is
introduced to minimize training time and computational memory. This maintains
the parser's efficiency in previously learned tasks while adapting it to new
tasks, mitigating the problem of catastrophic forgetting.
Overall, the thesis provides a comprehensive set of methods to improve
semantic parsing in resource-constrained conditions.Comment: PhD thesis, year of award 2023, 172 page
Few-Shot Semantic Parsing for New Predicates
In this work, we investigate the problems of semantic parsing in a few-shot
learning setting. In this setting, we are provided with utterance-logical form
pairs per new predicate. The state-of-the-art neural semantic parsers achieve
less than 25% accuracy on benchmark datasets when k= 1. To tackle this problem,
we proposed to i) apply a designated meta-learning method to train the model;
ii) regularize attention scores with alignment statistics; iii) apply a
smoothing technique in pre-training. As a result, our method consistently
outperforms all the baselines in both one and two-shot settings.Comment: Accepted to EACL 202
Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization
Human intelligence exhibits compositional generalization (i.e., the capacity
to understand and produce unseen combinations of seen components), but current
neural seq2seq models lack such ability. In this paper, we revisit iterative
back-translation, a simple yet effective semi-supervised method, to investigate
whether and how it can improve compositional generalization. In this work: (1)
We first empirically show that iterative back-translation substantially
improves the performance on compositional generalization benchmarks (CFQ and
SCAN). (2) To understand why iterative back-translation is useful, we carefully
examine the performance gains and find that iterative back-translation can
increasingly correct errors in pseudo-parallel data. (3) To further encourage
this mechanism, we propose curriculum iterative back-translation, which better
improves the quality of pseudo-parallel data, thus further improving the
performance.Comment: accepted in AAAI 202
- …