5,466 research outputs found
Performance Evaluation and Modeling of HPC I/O on Non-Volatile Memory
HPC applications pose high demands on I/O performance and storage capability.
The emerging non-volatile memory (NVM) techniques offer low-latency, high
bandwidth, and persistence for HPC applications. However, the existing I/O
stack are designed and optimized based on an assumption of disk-based storage.
To effectively use NVM, we must re-examine the existing high performance
computing (HPC) I/O sub-system to properly integrate NVM into it. Using NVM as
a fast storage, the previous assumption on the inferior performance of storage
(e.g., hard drive) is not valid any more. The performance problem caused by
slow storage may be mitigated; the existing mechanisms to narrow the
performance gap between storage and CPU may be unnecessary and result in large
overhead. Thus fully understanding the impact of introducing NVM into the HPC
software stack demands a thorough performance study.
In this paper, we analyze and model the performance of I/O intensive HPC
applications with NVM as a block device. We study the performance from three
perspectives: (1) the impact of NVM on the performance of traditional page
cache; (2) a performance comparison between MPI individual I/O and POSIX I/O;
and (3) the impact of NVM on the performance of collective I/O. We reveal the
diminishing effects of page cache, minor performance difference between MPI
individual I/O and POSIX I/O, and performance disadvantage of collective I/O on
NVM due to unnecessary data shuffling. We also model the performance of MPI
collective I/O and study the complex interaction between data shuffling,
storage performance, and I/O access patterns.Comment: 10 page
Like-sign Di-lepton Signals in Higgsless Models at the LHC
We study the potential LHC discovery of the Z1 KK gauge boson unitarizing
longitudinal W+W- scattering amplitude. In particular, we explore the decay
mode Z1->t tbar along with Z1-> W+W- without specifying the branching
fractions. We propose to exploit the associated production pp-> W Z1, and
select the final state of like-sign dileptons plus multijets and large missing
energy. We conclude that it is possible to observe the Z1 resonance at a 5
sigma level with an integrated luminosity of 100 inverse fb at the LHC upto 650
GeV for a dominant WW channel, and 560 GeV for a dominant ttbar channel.Comment: 13 pages, 7 figure
KPEval: Towards Fine-grained Semantic-based Evaluation of Keyphrase Extraction and Generation Systems
Despite the significant advancements in keyphrase extraction and keyphrase
generation methods, the predominant approach for evaluation only relies on
exact matching with human references and disregards reference-free attributes.
This scheme fails to recognize systems that generate keyphrases that are
semantically equivalent to the references or keyphrases that have practical
utility. To better understand the strengths and weaknesses of different
keyphrase systems, we propose a comprehensive evaluation framework consisting
of six critical dimensions: naturalness, faithfulness, saliency, coverage,
diversity, and utility. For each dimension, we discuss the desiderata and
design semantic-based metrics that align with the evaluation objectives.
Rigorous meta-evaluation studies demonstrate that our evaluation strategy
correlates better with human preferences compared to a range of previously used
metrics. Using this framework, we re-evaluate 18 keyphrase systems and further
discover that (1) the best model differs in different dimensions, with
pre-trained language models achieving the best in most dimensions; (2) the
utility in downstream tasks does not always correlate well with reference-based
metrics; and (3) large language models exhibit a strong performance in
reference-free evaluation
Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models
Keyphrase Generation (KPG) is a longstanding task in NLP with widespread
applications. The advent of sequence-to-sequence (seq2seq) pre-trained language
models (PLMs) has ushered in a transformative era for KPG, yielding promising
performance improvements. However, many design decisions remain unexplored and
are often made arbitrarily. This paper undertakes a systematic analysis of the
influence of model selection and decoding strategies on PLM-based KPG. We begin
by elucidating why seq2seq PLMs are apt for KPG, anchored by an
attention-driven hypothesis. We then establish that conventional wisdom for
selecting seq2seq PLMs lacks depth: (1) merely increasing model size or
performing task-specific adaptation is not parameter-efficient; (2) although
combining in-domain pre-training with task adaptation benefits KPG, it does
partially hinder generalization. Regarding decoding, we demonstrate that while
greedy search achieves strong F1 scores, it lags in recall compared with
sampling-based methods. Based on these insights, we propose DeSel, a
likelihood-based decode-select algorithm for seq2seq PLMs. DeSel improves
greedy search by an average of 4.7% semantic F1 across five datasets. Our
collective findings pave the way for deeper future investigations into
PLM-based KPG.Comment: EMNLP 2023 camera read
Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
Neural models that do not rely on pre-training have excelled in the keyphrase
generation task with large annotated datasets. Meanwhile, new approaches have
incorporated pre-trained language models (PLMs) for their data efficiency.
However, there lacks a systematic study of how the two types of approaches
compare and how different design choices can affect the performance of
PLM-based models. To fill in this knowledge gap and facilitate a more informed
use of PLMs for keyphrase extraction and keyphrase generation, we present an
in-depth empirical study. Formulating keyphrase extraction as sequence labeling
and keyphrase generation as sequence-to-sequence generation, we perform
extensive experiments in three domains. After showing that PLMs have
competitive high-resource performance and state-of-the-art low-resource
performance, we investigate important design choices including in-domain PLMs,
PLMs with different pre-training objectives, using PLMs with a parameter
budget, and different formulations for present keyphrases. Further results show
that (1) in-domain BERT-like PLMs can be used to build strong and
data-efficient keyphrase generation models; (2) with a fixed parameter budget,
prioritizing model depth over width and allocating more layers in the encoder
leads to better encoder-decoder models; and (3) introducing four in-domain
PLMs, we achieve a competitive performance in the news domain and the
state-of-the-art performance in the scientific domain.Comment: Technical Report. The contents are published in two separate papers
in EMNLP 2023 (arXiv:2310.06374) and LREC-COLING 2024 (arXiv:2402.14052
- …