128 research outputs found
Hierarchical Gated Recurrent Neural Tensor Network for Answer Triggering
In this paper, we focus on the problem of answer triggering ad-dressed by
Yang et al. (2015), which is a critical component for a real-world question
answering system. We employ a hierarchical gated recurrent neural tensor
(HGRNT) model to capture both the context information and the deep
in-teractions between the candidate answers and the question. Our result on F
val-ue achieves 42.6%, which surpasses the baseline by over 10 %
From Spelling to Grammar: A New Framework for Chinese Grammatical Error Correction
Chinese Grammatical Error Correction (CGEC) aims to generate a correct
sentence from an erroneous sequence, where different kinds of errors are mixed.
This paper divides the CGEC task into two steps, namely spelling error
correction and grammatical error correction. Specifically, we propose a novel
zero-shot approach for spelling error correction, which is simple but
effective, obtaining a high precision to avoid error accumulation of the
pipeline structure. To handle grammatical error correction, we design
part-of-speech (POS) features and semantic class features to enhance the neural
network model, and propose an auxiliary task to predict the POS sequence of the
target sentence. Our proposed framework achieves a 42.11 F0.5 score on CGEC
dataset without using any synthetic data or data augmentation methods, which
outperforms the previous state-of-the-art by a wide margin of 1.30 points.
Moreover, our model produces meaningful POS representations that capture
different POS words and convey reasonable POS transition rules
Evaluating the Capability of Large-scale Language Models on Chinese Grammatical Error Correction Task
Large-scale language models (LLMs) has shown remarkable capability in various
of Natural Language Processing (NLP) tasks and attracted lots of attention
recently. However, some studies indicated that large language models fail to
achieve promising result beyond the state-of-the-art models in English
grammatical error correction (GEC) tasks. In this report, we aim to explore the
how large language models perform on Chinese grammatical error correction tasks
and provide guidance for future work. We conduct experiments with 3 different
LLMs of different model scale on 4 Chinese GEC dataset. Our experimental
results indicate that the performances of LLMs on automatic evaluation metrics
falls short of the previous sota models because of the problem of
over-correction. Furthermore, we also discover notable variations in the
performance of LLMs when evaluated on different data distributions. Our
findings demonstrates that further investigation is required for the
application of LLMs on Chinese GEC task
Are Pre-trained Language Models Useful for Model Ensemble in Chinese Grammatical Error Correction?
Model ensemble has been in widespread use for Grammatical Error Correction
(GEC), boosting model performance. We hypothesize that model ensemble based on
the perplexity (PPL) computed by pre-trained language models (PLMs) should
benefit the GEC system. To this end, we explore several ensemble strategies
based on strong PLMs with four sophisticated single models. However, the
performance does not improve but even gets worse after the PLM-based ensemble.
This surprising result sets us doing a detailed analysis on the data and coming
up with some insights on GEC. The human references of correct sentences is far
from sufficient in the test data, and the gap between a correct sentence and an
idiomatic one is worth our attention. Moreover, the PLM-based ensemble
strategies provide an effective way to extend and improve GEC benchmark data.
Our source code is available at
https://github.com/JamyDon/PLM-based-CGEC-Model-Ensemble.Comment: 7 pages, 1 figure. Accepted by ACL 2023 (main conference, short
paper
- …