7 research outputs found
Chinese Named Entity Recognition Method for Domain-Specific Text
The Chinese named entity recognition (NER) is a critical task in natural language processing, aiming at identifying and classifying named entities in text. However, the specificity of domain texts and the lack of large-scale labelled datasets have led to the poor performance of NER methods trained on public domain corpora on domain texts. In this paper, a named entity recognition method incorporating sentence semantic information is proposed, mainly by adaptively incorporating sentence semantic information into character semantic information through an attention mechanism and a gating mechanism to enhance entity feature representation while attenuating the noise generated by irrelevant character information. In addition, to address the lack of large-scale labelled samples, we used data self-augmentation methods to expand the training samples. Furthermore, we introduced a Weighted Strategy considering that the low-quality samples generated by the data self-augmentation process can have a negative impact on the model. Experiments on the TCM prescriptions corpus showed that the F1 values of our method outperformed the comparison methods
Extensible Prompts for Language Models on Zero-shot Language Style Customization
We propose eXtensible Prompt (X-Prompt) for prompting a large language model
(LLM) beyond natural language (NL). X-Prompt instructs an LLM with not only NL
but also an extensible vocabulary of imaginary words. Registering new imaginary
words allows us to instruct the LLM to comprehend concepts that are difficult
to describe with NL words, thereby making a prompt more descriptive. Also,
these imaginary words are designed to be out-of-distribution (OOD) robust so
that they can be (re)used like NL words in various prompts, distinguishing
X-Prompt from soft prompt that is for fitting in-distribution data. We propose
context-augmented learning (CAL) to learn imaginary words for general
usability, enabling them to work properly in OOD (unseen) prompts. We
experiment X-Prompt for zero-shot language style customization as a case study.
The promising results of X-Prompt demonstrate its potential to facilitate
advanced interaction beyond the natural language interface, bridging the
communication gap between humans and LLMs.Comment: Accepted by NeurIPS 202
RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting
Large Language Models (LLMs) have demonstrated impressive capabilities in
creative tasks such as storytelling and E-mail generation. However, as LLMs are
primarily trained on final text results rather than intermediate revisions, it
might be challenging for them to perform text rewriting tasks. Most studies in
the rewriting tasks focus on a particular transformation type within the
boundaries of single sentences. In this work, we develop new strategies for
instruction tuning and reinforcement learning to better align LLMs for
cross-sentence rewriting tasks using diverse wording and structures expressed
through natural languages including 1) generating rewriting instruction data
from Wiki edits and public corpus through instruction generation and
chain-of-thought prompting; 2) collecting comparison data for reward model
training through a new ranking function. To facilitate this research, we
introduce OpenRewriteEval, a novel benchmark covers a wide variety of rewriting
types expressed through natural language instructions. Our results show
significant improvements over a variety of baselines. The public repository is
available on GitHub under Google Research
(https://github.com/google-research/google-research/tree/master/rewritelm)
Text Style Transfer: A Review and Experimental Evaluation
The stylistic properties of text have intrigued computational linguistics
researchers in recent years. Specifically, researchers have investigated the
Text Style Transfer (TST) task, which aims to change the stylistic properties
of the text while retaining its style independent content. Over the last few
years, many novel TST algorithms have been developed, while the industry has
leveraged these algorithms to enable exciting TST applications. The field of
TST research has burgeoned because of this symbiosis. This article aims to
provide a comprehensive review of recent research efforts on text style
transfer. More concretely, we create a taxonomy to organize the TST models and
provide a comprehensive summary of the state of the art. We review the existing
evaluation methodologies for TST tasks and conduct a large-scale
reproducibility study where we experimentally benchmark 19 state-of-the-art TST
algorithms on two publicly available datasets. Finally, we expand on current
trends and provide new perspectives on the new and exciting developments in the
TST field
Deep Learning for Text Style Transfer: A Survey
Text style transfer is an important task in natural language generation,
which aims to control certain attributes in the generated text, such as
politeness, emotion, humor, and many others. It has a long history in the field
of natural language processing, and recently has re-gained significant
attention thanks to the promising performance brought by deep neural models. In
this paper, we present a systematic survey of the research on neural text style
transfer, spanning over 100 representative articles since the first neural text
style transfer work in 2017. We discuss the task formulation, existing datasets
and subtasks, evaluation, as well as the rich methodologies in the presence of
parallel and non-parallel data. We also provide discussions on a variety of
important topics regarding the future development of this task. Our curated
paper list is at https://github.com/zhijing-jin/Text_Style_Transfer_SurveyComment: Computational Linguistics Journal 202