In the field of emotion estimation from voice, a commonly adopted approach involves extracting Mel-Frequency Cepstral Coefficients (MFCC) from audio signals. Recently, a method for emotion estimation based on Convolutional Neural Networks (CNNs) has been devised using MFCC images which can be obtained by converting MFCC data into image format. This method benefits from CNNs’ strong ability to capture local spatial patterns. Recently, however, Transformer-based models, originally developed for natural language processing tasks, have demonstrated outstanding performance in image recognition, surpassing CNNs in various classification benchmarks. Among them, Vision Transformer (ViT) has received particular attention for its ability to model long-range dependencies by applying self- attention mechanisms to image patches.This study investigates the classification accuracy and effectiveness of ViT-based methods in comparison to conventional CNN-based approaches, using image-based MFCC inputs derived from emotional speech. To enhance the performance of ViT under limited training data conditions, data augmentation and learning rate scheduling techniques are incorporated. While this introduces some differences in training conditions compared to previous CNN-based studies, the results demonstrate that ViT achieves superior emotion recognition performance. These findings suggest that ViT holds strong potential for improving the accuracy and robustness of audio-based emotion estimation systems.
Noda et al. (Wed,) studied this question.