From a Large Language Model to Three-Dimensional Sentiment

We present an automated model of sentiment which assigns to any arbitrary text three values: valence, arousal, and confidence (VAC) and is based on the three-dimensional framework of emotion introduced by Mehrabian and Russell [1]. While such a model is potentially valuable in any situation where a nuanced measurement of sentiment is important, our motivation was to quantify the dialog from psychological therapy and support sessions. The VAC scores are real-values that lie between -1 and 1, and thus the output lies in the 2×2×2 cube defined by the three dimensions. Internally, the model uses a convex combination of points in the cube with weights that are obtained from a publicly available zero-shot classifier built from the BART large language model (LLM) that has been fine-tuned on the Multi-Genre Natural Language Inference (MNLI) dataset. The classes used to produce weights for the convex combination were obtained through prompt engineering. In addition to describing the model and our approach to defining the classes, we show that the VAC model scores are strongly correlated with scores provided by human raters on individual words, and is arguably better than human on the dimension of confidence. By leveraging an LLM, it can process text of any size, is sensitive to subtlety and idiom, can be updated as language and technology evolve, and can produce meaningful results for any arbitrary sentence-length inputs. To illustrate a real-world application, we use the model to evaluate sentences spoken during a psychological therapy session