The traditional foreign language assessment system has the problems of single assessment dimension, subjective manual evaluation and lagging feedback, and the existing computer-aided assessment technology is limited to monomodal data analysis, which is difficult to fully reflect learners' actual communicative competence. Therefore, this study proposes an intelligent assessment framework for foreign language learning based on dynamic cross-modal attention network (DCAN), so as to achieve accurate fusion assessment of multimodal data. The end-to-end evaluation system of this framework takes learners' oral test videos as input, and outputs the evaluation results that meet CEFR standards through three core stages: multimodal feature extraction, dynamic cross-modal fusion and evaluation, and interpretable result generation. In the multi-modal feature extraction stage, Wav2Vec 2.0, BERT, MediaPipe Face Mesh and I3D models are used to extract the body language features of speech, text, facial movements and gestures. In the stage of dynamic cross-modal fusion and evaluation, DCAN combined with knowledge graph (KG) is used to dynamically adjust the weights of each modality to realize effective interaction and weighted fusion of multi-modal information, and the CEFR grade is obtained by mapping. In the interpretable result generation stage, the visual evaluation report synchronized with the video time axis is generated by using SHAP value analysis. The experimental results show that the root mean square error (RMSE) of DCAN model on the test set is 0.72, Pearson correlation coefficient (PCC) is 0.885, and the consistency accuracy rate is 58.9%, all of which are better than single-mode and other multi-mode fusion baseline models. In the evaluation of each dimension, especially in the non-verbal communication dimension (PCC reaches 0.91). The ablation experiment further confirmed the important contribution of visual mode, cultural KG and dynamic attention mechanism to the evaluation performance, and the SHAP value analysis provided clear and explanatory evaluation feedback without affecting the accuracy of the model. The intelligent assessment method combining NLP and CV proposed in this study effectively improves the comprehensiveness, accuracy and interpretability of foreign language learning assessment, and provides a new paradigm for the development of intelligent assessment in foreign language education.
Wang et al. (Sun,) studied this question.