Dual-rate parameter-efficient tuning with sample enhancement for source-free domain-adaptive keypoint detection

Abstract Source-free domain-adaptive keypoint detection (SFDA-KD) is a method that adapts a keypoint detection model to an unlabeled target domain without accessing the source domain data. In this task, keypoints are spread out across images, and the domain distribution shifts when labels are unavailable. Therefore, it is crucial to explore a broader global feature space and learn more robust features. However, a comprehensive solution that effectively addresses both challenges has yet to be developed. To this end, we propose a method termed dual-rate parameter-efficient tuning with adaptive augmentation (D-PETA). D-PETA consists of two learners with different learning rates based on the low-rank adaptation (LoRA) technique. A fast learner explores global features more quickly and avoids getting stuck in local minima, while a slow learner focuses on local features and stabilizes the training process. The two branches are interdependent, guiding each other to obtain more diverse and robust features. Furthermore, an adaptive augmentation module is introduced, which applies customized augmentations based on the uncertainty of the samples. This improvement leads to enhanced sample utilization and augmented model generalizability. Extensive experiments across diverse benchmarks, including the human body and hand datasets, demonstrate the effectiveness and generalizability of our proposed method.