New Conversations, New Evidence: Revisiting the Impact of Bots for Language Learning in the LLM Era

Generative AI and large language models (LLMs) have opened new opportunities to practice conversation meaningfully in a foreign language with a bot. I will revisit 40 years of research on dialogue-based computer-assisted language learning (CALL), before and after ChatGPT, to identify both how much it changed and what still holds up. I will explore the effectiveness of these “artificial” conversations to determine what works, for whom, for what and to what extent. In particular, I will share preliminary findings from a new meta-analysis of chatbots for language learning, looking at both “technological eras”. In this systematic review, we observe that the field is moving beyond comparing the technology, to comparing pedagogical and instructional conditions in their effects on language learning. I will call for more attention to task design and characteristics, and to scaffolding in conversation, particularly productive support for less proficient L2 speakers.