Delving Into the Devils of Knowledge Distillation for Object Detection: A Survey

ABSTRACT This survey provides a focused review of knowledge distillation (KD) techniques in object detection, a key area in computer vision. We categorise existing approaches into three primary types—feature‐based, relation‐based, and response‐based—each defined by the stage of knowledge transfer and the nature of information distilled. Beyond summarising these core paradigms, we examine their adaptations for emerging challenges such as continual learning, semi‐ and weakly‐supervised detection, and 3D object detection. We further conduct a systematic evaluation of representative methods on the COCO dataset, offering an in‐depth analysis of their strengths, limitations, and suitability across scenarios. A distinctive contribution of this work is its cross‐cutting synthesis of shared principles behind diverse KD strategies, revealing how these can be generalised and extended to new domains. We aim to provide researchers and practitioners with both a consolidated conceptual framework and actionable insights for advancing KD in object detection.