2 research outputs found
LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model
This paper describes our submission to subtask a and b of SemEval-2020 Task
4. For subtask a, we use a ALBERT based model with improved input form to pick
out the common sense statement from two statement candidates. For subtask b, we
use a multiple choice model enhanced by hint sentence mechanism to select the
reason from given options about why a statement is against common sense.
Besides, we propose a novel transfer learning strategy between subtasks which
help improve the performance. The accuracy scores of our system are 95.6 / 94.9
on official test set and rank 7 / 2 on Post-Evaluation
leaderboard.Comment: Accepted in SemEval2020. 7 pages, 4 figure
SemEval-2020 Task 4: Commonsense Validation and Explanation
In this paper, we present SemEval-2020 Task 4, Commonsense Validation and
Explanation (ComVE), which includes three subtasks, aiming to evaluate whether
a system can distinguish a natural language statement that makes sense to
humans from one that does not, and provide the reasons. Specifically, in our
first subtask, the participating systems are required to choose from two
natural language statements of similar wording the one that makes sense and the
one does not. The second subtask additionally asks a system to select the key
reason from three options why a given statement does not make sense. In the
third subtask, a participating system needs to generate the reason. We finally
attracted 39 teams participating at least one of the three subtasks. For
Subtask A and Subtask B, the performances of top-ranked systems are close to
that of humans. However, for Subtask C, there is still a relatively large gap
between systems and human performance. The dataset used in our task can be
found at https://github.com/wangcunxiang/SemEval2020-
Task4-Commonsense-Validation-and-Explanation; The leaderboard can be found at
https://competitions.codalab.org/competitions/21080#results.Comment: Task description paper of SemEval-2020 Task 4: Commonsense Validation
and Explanatio