This paper presents the design of a mid-air hand rehabilitation evaluation and virtual training system based on Leap Motion and gesture recognition algorithms. The system aims to provide a scientifically-grounded and accessible home-based rehabilitation solution for patients with hand injuries. It employs a spatio-temporal attention-enhanced multi-scale residual graph convolutional network algorithm for gesture recognition. Following this, specific joint angles are calculated and 14 representative gesture scores are derived. Weights are assigned to each scoring item via ridge regression to achieve a quantitative assessment. The rehabilitation training module comprises two modes: interactive turn-based games and music rhythm games, designed to train hand movement and dexterity. The system was subsequently tested to evaluate the performance of both the assessment function and the two training games. Results show an average gesture recognition accuracy of 94.86%. Furthermore, the reliability scores for both training games exceeded 90%. These findings demonstrate that the system achieves good accuracy in gesture recognition and effective assessment of hand rehabilitation progress. Keywords: Gesture recognition, hand rehabilitation, multi-scale residual graph convolution, attention mechanism, human-computer interaction
Zhou et al. (Mon,) studied this question.