224,915 research outputs found
A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation
We propose a normalization layer for unsupervised domain adaption in semantic
scene segmentation. Normalization layers are known to improve convergence and
generalization and are part of many state-of-the-art fully-convolutional neural
networks. We show that conventional normalization layers worsen the performance
of current Unsupervised Adversarial Domain Adaption (UADA), which is a method
to improve network performance on unlabeled datasets and the focus of our
research. Therefore, we propose a novel Domain Agnostic Normalization layer and
thereby unlock the benefits of normalization layers for unsupervised
adversarial domain adaptation. In our evaluation, we adapt from the synthetic
GTA5 data set to the real Cityscapes data set, a common benchmark experiment,
and surpass the state-of-the-art. As our normalization layer is domain agnostic
at test time, we furthermore demonstrate that UADA using Domain Agnostic
Normalization improves performance on unseen domains, specifically on
Apolloscape and Mapillary
To Normalize, or Not to Normalize: The Impact of Normalization on Part-of-Speech Tagging
Does normalization help Part-of-Speech (POS) tagging accuracy on noisy,
non-canonical data? To the best of our knowledge, little is known on the actual
impact of normalization in a real-world scenario, where gold error detection is
not available. We investigate the effect of automatic normalization on POS
tagging of tweets. We also compare normalization to strategies that leverage
large amounts of unlabeled data kept in its raw form. Our results show that
normalization helps, but does not add consistently beyond just word embedding
layer initialization. The latter approach yields a tagging model that is
competitive with a Twitter state-of-the-art tagger.Comment: In WNUT 201
Batch Layer Normalization, A new normalization layer for CNNs and RNN
This study introduces a new normalization layer termed Batch Layer
Normalization (BLN) to reduce the problem of internal covariate shift in deep
neural network layers. As a combined version of batch and layer normalization,
BLN adaptively puts appropriate weight on mini-batch and feature normalization
based on the inverse size of mini-batches to normalize the input to a layer
during the learning process. It also performs the exact computation with a
minor change at inference times, using either mini-batch statistics or
population statistics. The decision process to either use statistics of
mini-batch or population gives BLN the ability to play a comprehensive role in
the hyper-parameter optimization process of models. The key advantage of BLN is
the support of the theoretical analysis of being independent of the input data,
and its statistical configuration heavily depends on the task performed, the
amount of training data, and the size of batches. Test results indicate the
application potential of BLN and its faster convergence than batch
normalization and layer normalization in both Convolutional and Recurrent
Neural Networks. The code of the experiments is publicly available online
(https://github.com/A2Amir/Batch-Layer-Normalization).Comment: Published in proceedings of the 6th international conference on
Advances in Artificial Intelligence, ICAAI 2022, Birmingham, U
Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification
We introduce globally normalized convolutional neural networks for joint
entity classification and relation extraction. In particular, we propose a way
to utilize a linear-chain conditional random field output layer for predicting
entity types and relations between entities at the same time. Our experiments
show that global normalization outperforms a locally normalized softmax layer
on a benchmark dataset.Comment: EMNLP 201
- …