Instance-level data augmentation methods, exemplified by “copy-paste”, serve as a conventional strategy for improving the performance of small object detectors. The core idea involves leveraging background redundancy by compositing object instances with suitable backgrounds—drawn either from the same image or from different images—to increase both the quantity and diversity of training samples. However, existing methods often struggle with mismatches in background, scale, illumination, and viewpoint between instances and backgrounds. More critically, their predominant reliance on background information, without a joint understanding of instance-background characteristics, results in augmented images lacking visual realism. Empirical studies have demonstrated that such unrealistic images not only fail to improve detection performance but can even be detrimental. To tackle this problem, we propose a scene-understanding-driven approach that systematically addresses these mismatches via joint instance-background understanding. This is achieved through a unified framework that integrates image inpainting, image tagging, open-set object detection, the Segment Anything Model (SAM), and pose estimation to jointly model instance attributes, background semantics, and their interrelationships, thereby abandoning the random operation paradigm of existing methods and synthesizing highly realistic augmented images while preserving data diversity. On the VisDrone dataset, our method improves the mAP@0.5:0.95 and mAP@0.5 of the baseline detector by 1.6% and 2.2%, respectively. Both quantitative gains and qualitative visualizations confirm that the systematic resolution of these mismatches directly translates into significantly higher visual realism and detection performance improvements.
Li et al. (Fri,) studied this question.