Sequential labeling tasks necessitate the computation of sentence
representations for each word within a given sentence. With the advent of
advanced pretrained language models; one common approach involves incorporating
a BiLSTM layer to bolster the sequence structure information at the output
level. Nevertheless, it has been empirically demonstrated (P.-H. Li et al.,
2020) that the potential of BiLSTM for generating sentence representations for
sequence labeling tasks is constrained, primarily due to the amalgamation of
fragments form past and future sentence representations to form a complete
sentence representation. In this study, we discovered that strategically
integrating the whole sentence representation, which existing in the first cell
and last cell of BiLSTM, into sentence representation of ecah cell, could
markedly enhance the F1 score and accuracy. Using BERT embedded within BiLSTM
as illustration, we conducted exhaustive experiments on nine datasets for
sequence labeling tasks, encompassing named entity recognition (NER), part of
speech (POS) tagging and End-to-End Aspect-Based sentiment analysis (E2E-ABSA).
We noted significant improvements in F1 scores and accuracy across all examined
datasets