This paper focuses on developing an AI game assistant, initially targeting Genshin Impact. The training data consists of a conversational dataset and a knowledge dataset, collected from the player’s perspective and through web scraping from game-related wikis, especially character information. The AI assistant architecture integrates three components: (1) LLaMA 3 Taiwan, a large language model for player interaction; (2) VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech), a TTS (text-to-speech) system that generates the assistant’s voice; and (3) Unreal Engine 5, which animates the assistant character. To build the language model, a Linux environment is configured using Windows Subsystem for Linux 2 (WSL 2), offering fast, lightweight virtualization and easy file access. Unsloth is used to reduce GPU usage and accelerate fine-tuning of Llama 3, enabling the model to answer in-game questions effectively. Finally, VITS generates speech synchronized with the animated character in Unreal Engine 5, completing the system. The result is an interactive AI assistant capable of responding with contextual dialogue, voice output, and real-time animation, enhancing the game experience.
Chen et al. (Mon,) studied this question.