Few-shot sign language recognition using contrastive learning

Background Sign languages, which are utilized by millions of deaf or hearing-impaired individuals globally, offer unique visual communication that varies across cultures. Yet, most current communication technologies are primarily designed for spoken languages, creating barriers for deaf people. Objective This study presents a novel sign language recognition method that learns from a few samples and requires no retraining for new signs, leveraging a unique approach inspired by the face recognition domain. Methods Using contrastive learning, a deep learning technique for representation learning, the model embeds each sign’s holistic representation, by clustering similar signs and distancing the dissimilar ones in the feature space. Results The method was evaluated on custom Multi-Examples Sign Language and Few-shot Sign Language datasets and outperformed state-of-the-art methods with an accuracy of 97.3% (an improvement of 6%). Conclusions This paper develops a novel method that integrates a contrastive learning approach tailored to address the unique challenges of this area. It also identified two key parameters (on the model performance) in the sign recognition domain: the impact of sample size and the influence of different regions of the signer’s body. This research offers an efficient solution to improve deaf people’s interaction with modern technologies.