Using DistilBERT to assign HS codes to international trading transactions

One significant source of national revenue for many countries is the tax levied on international trade. Tax collection can be achieved by accurately classifying international trading commodities according to Harmonised System (HS) codes, which later can be used to impose customs duty/tax rates. The current approach to assigning HS codes to transactions relies on HS codes filled out by international traders and being manually inspected by customs officers. This ap-proach is tedious and prone to error, potentially leading to fraudulent activity. However, commodity texts are hard to classify because of their short length, noise, ambiguity, and use of a lot of technical terms. To address these challenges, our research aims to determine the HS codes automatically from commodity de-scription texts in trading transactions using text classification techniques. This paper proposes utilising transformers models, BERT and its variants, Distil-BERT, which is claimed to be lighter and faster than the BERT model and has the advantage of being deployed in computational resource-constrained environ-ments. The proposed approach adopts a transfer learning procedure to perform fine-tuning hyperparameters of BERT and DistilBERT. It is evaluated using real-world customs data for multi-class classification of commodity transactions in international trading. Experimental results indicate that both models achieve a comparable performance result