2 research outputs found
Applying a Pre-trained Language Model to Spanish Twitter Humor Prediction
Our entry into the HAHA 2019 Challenge placed in the classification
task and in the regression task. We describe our system and
innovations, as well as comparing our results to a Naive Bayes baseline. A
large Twitter based corpus allowed us to train a language model from scratch
focused on Spanish and transfer that knowledge to our competition model. To
overcome the inherent errors in some labels we reduce our class confidence with
label smoothing in the loss function. All the code for our project is included
in a GitHub repository for easy reference and to enable replication by others.Comment: IberLEF 2019 Worksho
LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading
In this paper, we assess the ability of BERT and its derivative models
(RoBERTa, DistilBERT, and ALBERT) for short-edits based humor grading. We test
these models for humor grading and classification tasks on the Humicroedit and
the FunLines dataset. We perform extensive experiments with these models to
test their language modeling and generalization abilities via zero-shot
inference and cross-dataset inference based approaches. Further, we also
inspect the role of self-attention layers in humor-grading by performing a
qualitative analysis over the self-attention weights from the final layer of
the trained BERT model. Our experiments show that all the pre-trained BERT
derivative models show significant generalization capabilities for
humor-grading related tasks.Comment: Submitted at SemEval-2020 worksho