This paper presents an analysis regarding an influence of the Distance Metric
Learning (DML) loss functions on the supervised fine-tuning of the language
models for classification tasks. We experimented with known datasets from
SentEval Transfer Tasks.
Our experiments show that applying the DML loss function can increase
performance on downstream classification tasks of RoBERTa-large models in
few-shot scenarios. Models fine-tuned with the use of SoftTriple loss can
achieve better results than models with a standard categorical cross-entropy
loss function by about 2.89 percentage points from 0.04 to 13.48 percentage
points depending on the training dataset. Additionally, we accomplished a
comprehensive analysis with explainability techniques to assess the models'
reliability and explain their results