A Study on the Application of Vision-Language Models for Zero-Shot Obstruction Detection in Drone Imagery

Key Points

• The primary aim is to improve object detection capabilities beyond labeled classes using vision-language models.
• Propose a zero-shot object detection technique using vision-language models.
• Replace text embeddings with visual embeddings extracted from reference images.
• Utilize a CNN-based binary classifier for post-processing to enhance detection precision.
• Experiment on a dataset featuring 10 classes of obstructions in high-resolution drone imagery.
• Demonstrated up to a 60% improvement in detection accuracy for 8 out of 10 classes compared to text-based methods.
• Proven effective particularly in context-dependent object detection.
• Potential for integrating hybrid structures with text queries for better versatility.

Abstract