AI4ASC: A Hierarchical Multi-Level AI-Driven Framework for Gesture Analysis and Monitoring in ASC Children

Autism is characterized by differences in communication and social interaction, where nonverbal behaviors such as gestures provide valuable indicators for understanding social communication development. Traditional gesture assessment methods heavily rely on manual video review, which is labor-intensive, error-prone, and lacks scalability. Artificial Intelligence (AI) is a powerful solution to enhance the accuracy of autism diagnosis and therapies by enabling continuous, objective analysis of children nonverbal behaviors, such as gestures, thereby improving the assessment of their social and communication skills. This work leverages computer vision for gesture recognition in autistic children diagnosis, proposing AI4ASC, a hierarchical, multi (four)-level AI framework for the automated analysis and monitoring of gestures in autistic children. It focuses on pointing gesture, one of the early indicators of social communication development, to identify behaviors in children that, according to clinical evidence, behave differently to neurotypical ones. Experimental results on real-world clinical videos show that AI4ASC achieves high performance across all system levels: child detection with 88.6% mAP, hand detection with 95.7% of precision, and pointing gesture classification with 97.4% of overall precision. The final clustering level consolidates detected gestures with precision and quantifies them for clinician-guided interpretation. A user-friendly graphical interface and a client–server architecture enable therapist interaction and validation, supporting the integration of AI4ASC into clinical workflows for behavioral assessment.