PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
October 20, 20251 citationsOpen Access

Code-switching Speech Recognition Under the Lens: Model- and Data-Centric Perspectives

View Full Paper
HLHexin LiuHZHaoyang ZhangQZQiquan Zhang

Key Points

  • Code-switching automatic speech recognition faces challenges from accent bias and language confusion, complicating data interpretation.
  • The study compares algorithm methods like language-aware multi-task learning, showing varying effectiveness based on linguistic characteristics.
  • Data augmentation via TTS significantly impacts ASR performance, improving the recognition of code-switching through enhanced speech-text pairs.
  • The simplified equivalence constraint theory (SECT) generates text better resembling real-world language use, enhancing linguistic validity.

Abstract

Code-switching automatic speech recognition (CS-ASR) presents unique challenges due to language confusion introduced by spontaneous intra-sentence switching and accent bias that blurs the phonetic boundaries. Although the constituent languages may be individually high-resource, the scarcity of annotated code-switching data further compounds these challenges. In this paper, we systematically analyze CS-ASR from both model-centric and data-centric perspectives. By comparing state-of-the-art algorithmic methods, including language-specific processing and auxiliary language-aware multi-task learning, we discuss their varying effectiveness across datasets with different linguistic characteristics. On the data side, we first investigate TTS as a data augmentation method. By varying the textual characteristics and speaker accents, we analyze the impact of language confusion and accent bias on CS-ASR. To further mitigate data scarcity and enhance textual diversity, we propose a prompting strategy by simplifying the equivalence constraint theory (SECT) to guide large language models (LLMs) in generating linguistically valid code-switching text. The proposed SECT outperforms existing methods in ASR performance and linguistic quality assessments, generating code-switching text that more closely resembles real-world code-switching text. When used to generate speech-text pairs via TTS, SECT proves effective in improving CS-ASR performance. Our analysis of both model- and data-centric methods underscores that effective CS-ASR requires strategies to be carefully aligned with the specific linguistic characteristics of the code-switching data.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Liu et al. (2025) studied this question.

synapsesocial.com/papers/68f5fcce8d54a28a75cf1d45https://doi.org/10.48550/arxiv.2509.24310
Ask AI
Helpful
Bookmark
Share
View Full Paper