What question did this study set out to answer?

The aim is to compare the effectiveness of various Transformer models in classifying sentiment and emotions from social media data.

March 16, 2026Open Access

A Comparative Analysis of Transformer Architectures for Sentiment and Emotion Classification

Key Points

The aim is to compare the effectiveness of various Transformer models in classifying sentiment and emotions from social media data.
Analyzed four Transformer models: BERT, ALBERT, T5, and XLNet.
Employed two datasets: Twitter data and an Emotion dataset with labeled categories.
Trained models under consistent preprocessing and testing settings for varying epochs.
BERT achieved maximum accuracies of 88.63% and 97.05% for the X and Emotion datasets respectively.
XLNet showed strong potential for handling long-range dependencies.
ALBERT delivered balanced performance due to its lightweight design.
T5 performed worse compared to other models.

Abstract

This paper is a comparison of four Transformer model (BERT, ALBERT, T5, and XLNet) in the case of sentiment analysis of tasks based on data from social media. Used were two data sets: the X (Twitter) data set with tweets of messages related to games and the Emotion data set labeled in the anger, joy, and fear categories. Trained was the model in the same settings of preprocessing, training, as well as in the test settings for 3, 5, 7, and 10 epochs. Presented were results that indicated that the accuracy increased with the higher number of epochs. The maximum accuracies occurred in the case of the BERT model—88.63% in the case of the X data set as well as in the case of the Emotion data set, 97.05%. XLNet established great potential for long-range dependencies, and ALBERT obtained balanced performance due to lightweight architecture. On the contrary, performance of T5 was less in comparison to others. Generally, it could be inferred that Transformer architecture is superior to traditional machine learning technique in the context of sentiment analysis due to higher accuracy and better contextual understanding.

Mark Helpful

Bookmark

Relay

View Full Paper