1,259 research outputs found
Fine-tuning Transformer-based Encoder for Turkish Language Understanding Tasks
Deep learning-based and lately Transformer-based language models have been
dominating the studies of natural language processing in the last years. Thanks
to their accurate and fast fine-tuning characteristics, they have outperformed
traditional machine learning-based approaches and achieved state-of-the-art
results for many challenging natural language understanding (NLU) problems.
Recent studies showed that the Transformer-based models such as BERT, which is
Bidirectional Encoder Representations from Transformers, have reached
impressive achievements on many tasks. Moreover, thanks to their transfer
learning capacity, these architectures allow us to transfer pre-built models
and fine-tune them to specific NLU tasks such as question answering. In this
study, we provide a Transformer-based model and a baseline benchmark for the
Turkish Language. We successfully fine-tuned a Turkish BERT model, namely
BERTurk that is trained with base settings, to many downstream tasks and
evaluated with a the Turkish Benchmark dataset. We showed that our studies
significantly outperformed other existing baseline approaches for Named-Entity
Recognition, Sentiment Analysis, Question Answering and Text Classification in
Turkish Language. We publicly released these four fine-tuned models and
resources in reproducibility and with the view of supporting other Turkish
researchers and applications
How to Fine-Tune BERT for Text Classification?
Language model pre-training has proven to be useful in learning universal
language representations. As a state-of-the-art language model pre-training
model, BERT (Bidirectional Encoder Representations from Transformers) has
achieved amazing results in many language understanding tasks. In this paper,
we conduct exhaustive experiments to investigate different fine-tuning methods
of BERT on text classification task and provide a general solution for BERT
fine-tuning. Finally, the proposed solution obtains new state-of-the-art
results on eight widely-studied text classification datasets
- …