A User Recognition Methodology Based on Voice Biometrics and Dynamic Clustering for Social Robots

Social robots are systems designed to assist people across different fields. During their operation, they have to interact with people with different characteristics and necessities. Consequently, correctly recognising the user interacting with the robot facilitates the generation of a personalised experience that satisfies the user’s needs. In robotics, user recognition is typically based on face recognition from image processing and datasets that require retraining the network to include new users. However, some robots, such as pet-like companions, often lack a camera due to reduced dimensions, limited computational resources, or privacy constraints. Additionally, robots can occasionally encounter new users, requiring online recognition to provide a personalised interaction experience. To address these limitations, this article presents a user recognition system based on voice biometrics and dynamic clustering for adaptive social robots. We evaluate a set of open-source models for voice biometric extraction using different clustering algorithms to identify the best combination for our application. The resulting system is implemented in a pet-like robot companion that is used for the affective support of older adults, demonstrating its capacities in a real-world scenario. The system achieves more than 73% accuracy in recognising users who had previously spoken to the robot and more than 71% success in recognising new users who had not previously interacted with the robot and creating a personal profile for them. However, the system still detects noise, especially when the speaker has never interacted with the robot.