1,370 research outputs found
The effect of primer choice and short read sequences on the outcome of 16S rRNA gene based diversity studies
Different regions of the bacterial 16S rRNA gene evolve at different evolutionary rates. The scientific outcome of short read sequencing studies therefore alters with the gene region sequenced. We wanted to gain insight in the impact of primer choice on the outcome of short read sequencing efforts. All the unknowns associated with sequencing data, i.e. primer coverage rate, phylogeny, OTU-richness and taxonomic assignment, were therefore implemented in one study for ten well established universal primers (338f/r, 518f/r, 799f/r, 926f/r and 1062f/r) targeting dispersed regions of the bacterial 16S rRNA gene. All analyses were performed on nearly full length and in silico generated short read sequence libraries containing 1175 sequences that were carefully chosen as to present a representative substitute of the SILVA SSU database. The 518f and 799r primers, targeting the V4 region of the 16S rRNA gene, were found to be particularly suited for short read sequencing studies, while the primer 1062r, targeting V6, seemed to be least reliable. Our results will assist scientists in considering whether the best option for their study is to select the most informative primer, or the primer that excludes interferences by host-organelle DNA. The methodology followed can be extrapolated to other primers, allowing their evaluation prior to the experiment
An Integrated Method for Determination of the Oswald Factor in a Multi-Fidelity Design Environment
Aircraft conceptual design often focuses on unconventional
configurations like for example forward
swept wings. Assessing the characteristics
of these configurations usually requires the use
of physic based analysis modules. This is due
to the fact that for unconventional configurations
no sucient database for historic based analysis
modules is available.
Nevertheless, physic based models require a
lot of input data and their computational cost can
be high. Generating input values in a trade study
manually is work-intensive and error-prone.
Conceptual design modules can be used to
generate sucient input data for physic based
models and their results can be re-integrated into
the conceptual design phase. In this study a direct
link between a conceptual design module
and an aerodynamic design module is presented.
Geometric information is generated by the conceptual
design module and the physic based results,
in form of the Oswald factor, are then fed
back.
Apart from the direct link, an equation for determination
of the Oswald factor is derived via a
Symbolic Regression Approach
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale
We study the zero-shot transfer capabilities of text matching models on a
massive scale, by self-supervised training on 140 source domains from community
question answering forums in English. We investigate the model performances on
nine benchmarks of answer selection and question similarity tasks, and show
that all 140 models transfer surprisingly well, where the large majority of
models substantially outperforms common IR baselines. We also demonstrate that
considering a broad selection of source domains is crucial for obtaining the
best zero-shot transfer performances, which contrasts the standard procedure
that merely relies on the largest and most similar domains. In addition, we
extensively study how to best combine multiple source domains. We propose to
incorporate self-supervised with supervised multi-task learning on all
available source domains. Our best zero-shot transfer model considerably
outperforms in-domain BERT and the previous state of the art on six benchmarks.
Fine-tuning of our model with in-domain data results in additional large gains
and achieves the new state of the art on all nine benchmarks.Comment: EMNLP-202
CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models
While many languages possess processes of joining two or more words to create
compound words, previous studies have been typically limited only to languages
with excessively productive compound formation (e.g., German, Dutch) and there
is no public dataset containing compound and non-compound words across a large
number of languages. In this work, we systematically study decompounding, the
task of splitting compound words into their constituents, at a wide scale. We
first address the data gap by introducing a dataset of 255k compound and
non-compound words across 56 diverse languages obtained from Wiktionary. We
then use this dataset to evaluate an array of Large Language Models (LLMs) on
the decompounding task. We find that LLMs perform poorly, especially on words
which are tokenized unfavorably by subword tokenization. We thus introduce a
novel methodology to train dedicated models for decompounding. The proposed
two-stage procedure relies on a fully self-supervised objective in the first
stage, while the second, supervised learning stage optionally fine-tunes the
model on the annotated Wiktionary data. Our self-supervised models outperform
the prior best unsupervised decompounding models by 13.9% accuracy on average.
Our fine-tuned models outperform all prior (language-specific) decompounding
tools. Furthermore, we use our models to leverage decompounding during the
creation of a subword tokenizer, which we refer to as CompoundPiece.
CompoundPiece tokenizes compound words more favorably on average, leading to
improved performance on decompounding over an otherwise equivalent model using
SentencePiece tokenization.Comment: EMNLP 202
Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields
We compare different models for low resource multi-task sequence tagging that
leverage dependencies between label sequences for different tasks. Our analysis
is aimed at datasets where each example has labels for multiple tasks. Current
approaches use either a separate model for each task or standard multi-task
learning to learn shared feature representations. However, these approaches
ignore correlations between label sequences, which can provide important
information in settings with small training datasets. To analyze which
scenarios can profit from modeling dependencies between labels in different
tasks, we revisit dynamic conditional random fields (CRFs) and combine them
with deep neural networks. We compare single-task, multi-task and dynamic CRF
setups for three diverse datasets at both sentence and document levels in
English and German low resource scenarios. We show that including silver labels
from pretrained part-of-speech taggers as auxiliary tasks can improve
performance on downstream tasks. We find that especially in low-resource
scenarios, the explicit modeling of inter-dependencies between task predictions
outperforms single-task as well as standard multi-task models
Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation
Many NLP pipelines split text into sentences as one of the crucial
preprocessing steps. Prior sentence segmentation tools either rely on
punctuation or require a considerable amount of sentence-segmented training
data: both central assumptions might fail when porting sentence segmenters to
diverse languages on a massive scale. In this work, we thus introduce a
multilingual punctuation-agnostic sentence segmentation method, currently
covering 85 languages, trained in a self-supervised fashion on unsegmented
text, by making use of newline characters which implicitly perform segmentation
into paragraphs. We further propose an approach that adapts our method to the
segmentation in a given corpus by using only a small number (64-256) of
sentence-segmented examples. The main results indicate that our method
outperforms all the prior best sentence-segmentation tools by an average of
6.1% F1 points. Furthermore, we demonstrate that proper sentence segmentation
has a point: the use of a (powerful) sentence segmenter makes a considerable
difference for a downstream application such as machine translation (MT). By
using our method to match sentence segmentation to the segmentation used during
training of MT models, we achieve an average improvement of 2.3 BLEU points
over the best prior segmentation tool, as well as massive gains over a trivial
segmenter that splits text into equally sized blocks.Comment: ACL 202
Modular and Parameter-efficient Fine-tuning of Language Models
Transfer learning has recently become the dominant paradigm of natural language processing. Models pre-trained on unlabeled data can be fine-tuned for downstream tasks based on only a handful of examples. A long-term goal is to develop models that acquire new information at scale without incurring negative transfer and that generalize systematically to new settings. Modular deep learning has emerged as a promising solution to these challenges, by updating parameter-efficient units of computation locally and asynchronously. These units are often implemented as modules that are interlaid between layers, interpolated with pre-trained parameters, or concatenated to the inputs. Conditioned on tasks or examples, information is routed to multiple modules through a fixed or learned function, followed by an aggregation of their outputs. This property enables compositional generalization, by disentangling knowledge and recombining it in new ways.
In this thesis, we provide a unified view of modularity in natural language processing, spanning across four dimensions; specifically, we disentangle modularity into computation functions, routing functions, aggregation functions, and the training setting. Along those axes, we propose multiple contributions: a research framework which encompasses all dimensions; a novel attention-based aggregation function which combines the knowledge stored within different modules; routing mechanisms for out of distribution generalization in cross-lingual transfer scenarios; a dataset and modular training strategies for multimodal and multilingual transfer learning; a modular pre-training strategy to tackle catastrophic interference of heterogeneous data
A neural autoencoder approach for document ranking and query refinement in pharmacogenomic information retrieval
In this study, we investigate learning-to-
rank and query refinement approaches for
information retrieval in the pharmacogenomic domain. The goal is to improve the
information retrieval process of biomedical curators, who manually build knowledge bases for personalized medicine. We
study how to exploit the relationships be-
tween genes, variants, drugs, diseases and
outcomes as features for document ranking and query refinement.
For a supervised approach, we are faced with a
small amount of annotated data and a large
amount of unannotated data. Therefore,
we explore ways to use a neural document
auto-encoder in a semi-supervised approach. We show that a combination of established algorithms, feature-engineering
and a neural auto-encoder model yield
promising results in this setting
- âŠ