43 research outputs found
ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task
Transformer-based language models, including ChatGPT, have demonstrated
exceptional performance in various natural language generation tasks. However,
there has been limited research evaluating ChatGPT's keyphrase generation
ability, which involves identifying informative phrases that accurately reflect
a document's content. This study seeks to address this gap by comparing
ChatGPT's keyphrase generation performance with state-of-the-art models, while
also testing its potential as a solution for two significant challenges in the
field: domain adaptation and keyphrase generation from long documents. We
conducted experiments on six publicly available datasets from scientific
articles and news domains, analyzing performance on both short and long
documents. Our results show that ChatGPT outperforms current state-of-the-art
models in all tested datasets and environments, generating high-quality
keyphrases that adapt well to diverse domains and document lengths
Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extraction
Keyphrase extraction (KPE) is an important task in Natural Language
Processing for many scenarios, which aims to extract keyphrases that are
present in a given document. Many existing supervised methods treat KPE as
sequential labeling, span-level classification, or generative tasks. However,
these methods lack the ability to utilize keyphrase information, which may
result in biased results. In this study, we propose Diff-KPE, which leverages
the supervised Variational Information Bottleneck (VIB) to guide the text
diffusion process for generating enhanced keyphrase representations. Diff-KPE
first generates the desired keyphrase embeddings conditioned on the entire
document and then injects the generated keyphrase embeddings into each phrase
representation. A ranking network and VIB are then optimized together with rank
loss and classification loss, respectively. This design of Diff-KPE allows us
to rank each candidate phrase by utilizing both the information of keyphrases
and the document. Experiments show that Diff-KPE outperforms existing KPE
methods on a large open domain keyphrase extraction benchmark, OpenKP, and a
scientific domain dataset, KP20K.Comment: 10 pages, 2 figure
Keyphrase Extraction from Disaster-related Tweets
While keyphrase extraction has received considerable attention in recent
years, relatively few studies exist on extracting keyphrases from social media
platforms such as Twitter, and even fewer for extracting disaster-related
keyphrases from such sources. During a disaster, keyphrases can be extremely
useful for filtering relevant tweets that can enhance situational awareness.
Previously, joint training of two different layers of a stacked Recurrent
Neural Network for keyword discovery and keyphrase extraction had been shown to
be effective in extracting keyphrases from general Twitter data. We improve the
model's performance on both general Twitter data and disaster-related Twitter
data by incorporating contextual word embeddings, POS-tags, phonetics, and
phonological features. Moreover, we discuss the shortcomings of the often used
F1-measure for evaluating the quality of predicted keyphrases with respect to
the ground truth annotations. Instead of the F1-measure, we propose the use of
embedding-based metrics to better capture the correctness of the predicted
keyphrases. In addition, we also present a novel extension of an
embedding-based metric. The extension allows one to better control the penalty
for the difference in the number of ground-truth and predicted keyphrasesComment: 12 pages, 7 figure
Automatic Metadata Extraction Incorporating Visual Features from Scanned Electronic Theses and Dissertations
Electronic Theses and Dissertations (ETDs) contain domain knowledge that can
be used for many digital library tasks, such as analyzing citation networks and
predicting research trends. Automatic metadata extraction is important to build
scalable digital library search engines. Most existing methods are designed for
born-digital documents, so they often fail to extract metadata from scanned
documents such as for ETDs. Traditional sequence tagging methods mainly rely on
text-based features. In this paper, we propose a conditional random field (CRF)
model that combines text-based and visual features. To verify the robustness of
our model, we extended an existing corpus and created a new ground truth corpus
consisting of 500 ETD cover pages with human validated metadata. Our
experiments show that CRF with visual features outperformed both a heuristic
and a CRF model with only text-based features. The proposed model achieved
81.3%-96% F1 measure on seven metadata fields. The data and source code are
publicly available on Google Drive (https://tinyurl.com/y8kxzwrp) and a GitHub
repository (https://github.com/lamps-lab/ETDMiner/tree/master/etd_crf),
respectively.Comment: 7 pages, 4 figures, 1 table. Accepted by JCDL '21 as a short pape
A Survey on Semantic Processing Techniques
Semantic processing is a fundamental research domain in computational
linguistics. In the era of powerful pre-trained language models and large
language models, the advancement of research in this domain appears to be
decelerating. However, the study of semantics is multi-dimensional in
linguistics. The research depth and breadth of computational semantic
processing can be largely improved with new technologies. In this survey, we
analyzed five semantic processing tasks, e.g., word sense disambiguation,
anaphora resolution, named entity recognition, concept extraction, and
subjectivity detection. We study relevant theoretical research in these fields,
advanced methods, and downstream applications. We connect the surveyed tasks
with downstream applications because this may inspire future scholars to fuse
these low-level semantic processing tasks with high-level natural language
processing tasks. The review of theoretical research may also inspire new tasks
and technologies in the semantic processing domain. Finally, we compare the
different semantic processing techniques and summarize their technical trends,
application trends, and future directions.Comment: Published at Information Fusion, Volume 101, 2024, 101988, ISSN
1566-2535. The equal contribution mark is missed in the published version due
to the publication policies. Please contact Prof. Erik Cambria for detail