A Multi-modal Hierarchical Approach for Chinese Spelling Correction Using Multi-Head Attention and Residual Connec-tions

Abstract The primary objective of Chinese Spelling Correction is to detect and correct erroneous characters within Chinese text, which can result from various factors, such as inaccuracies in pinyin representation, character resemblance, and semantic discrepancies. However, existing methods often struggle to fully address these types of errors, impacting the overall correction accuracy. This paper introduces a multi-modal feature encoder designed to efficiently extract features from three distinct modalities: pinyin, semantics, and character morphology. Unlike previous methods that rely on direct fusion or fixed-weight summation to integrate multi-modal information, our approach employs a multi-head attention mechanism to focused more on relevant modal information while disregarding less pertinent data. To prevent issues such as gradient explosion or vanishing, the model incorporates a residual connection of the original text vector for fine-tuning. This approach ensures robust model performance by maintaining essential linguistic details throughout the correction process. Experimental evaluations on the SIGHAN benchmark dataset demonstrate that our model outperforms baseline approaches across various metrics and datasets, confirming its effectiveness and feasibility.