Recent studies have provided empirical evidence of the wide-ranging potential
of Generative Pre-trained Transformer (GPT), a pretrained language model, in
the field of natural language processing. GPT has been effectively employed as
a decoder within state-of-the-art (SOTA) question answering systems, yielding
exceptional performance across various tasks. However, the current research
landscape concerning GPT's application in Vietnamese remains limited. This
paper aims to address this gap by presenting an implementation of GPT-2 for
community-based question answering specifically focused on COVID-19 related
queries in Vietnamese. We introduce a novel approach by conducting a
comparative analysis of different Transformers vs SOTA models in the
community-based COVID-19 question answering dataset. The experimental findings
demonstrate that the GPT-2 models exhibit highly promising outcomes,
outperforming other SOTA models as well as previous community-based COVID-19
question answering models developed for Vietnamese