RG-Hybrid: pure late fusion of RoBERTa and a graph transformer for robust, interpretable sentiment classification

Abstract Large pretrained language models (PLMs) provide strong semantic signals for sentiment classification but process text as linear token streams, leaving syntactic relations implicit. This limits robustness on structure-sensitive phenomena such as negation scope, coordination, and clause-level dependencies. Graph neural networks can encode such relations explicitly but lack the broad semantic grounding of large-scale pretraining. Early- and mid-fusion hybrids address this gap but blend modalities through learned parameters, obscuring individual contributions and risking overfitting on smaller datasets. We introduce RG-Hybrid , a dual-branch architecture that combines a fine-tuned RoBERTa encoder with a graph transformer operating over dependency parses. The two branches are trained separately and merged only at the decision stage through parameter-free logit averaging. This design keeps the semantic and structural contributions fully separable, enabling direct attribution of each branch’s role in the final prediction. Evaluated on four sentiment benchmarks spanning document-level and sentence-level corpora, RG-Hybrid achieves consistent and statistically significant improvements over a fine-tuned RoBERTa-only baseline, with relative error rate reductions of up to 17.66%. It outperforms transductive PLM–GNN baselines despite operating fully inductively. Qualitative analysis confirms that explicit dependency structure resolves polarity ambiguities introduced by negation and concessive constructions, while also identifying failure modes on ironic text.