Edge-refined U-net for multi-task weed detection and segmentation with pixel-level boundary refinement

Weed detection and segmentation, particularly in the presence of multiple weed classes, present critical challenges for spraying robots in crop fields. Traditional weed classification models are limited to categorizing individual weeds, such as grasses or broadleaf weeds. Existing deep learning techniques have been proposed for selective fields or field-dependent models that use rough outlines with YOLO annotations, which often lack clear boundaries. These result in errors at the edges, where precision is most critical. To overcome these challenges, this research proposed a multi-class weed-detection and segmentation model that can identify all weed types across any field. The practical and scalable nature of this model makes it a unique contribution and a real-time solution for complex weed fields. The present study proposes a novel semantic segmentation framework utilizing an edge-refined enhanced U-Net and a deep learning model for pixel-level analysis of weeds and crops. The input data is pre-processed by extracting vegetation regions using an Excess Green (ExG) and Otsu thresholding approach, effectively suppressing irrelevant soil background. The edge-refined U–Net–based model is trained for three-class segmentation as multi-subclass weed segmentation, crop, and background, which is enhanced with auxiliary edge supervision to improve boundary refinement. This proposed model is a multitask model trained and developed on the complex 16-class weed dataset (MH-WEED16) from Intel RealSense field imagery. The model is further tested on an unseen Sorghum weed dataset and real-time agricultural images. Experimental results show that the proposed model achieves robust performance across multiple evaluation metrics like Dice score of 80 %, IoU of 70 %, boundary IoU of 72 %, and Hausdorff distance as 39 for automated weed–crop discrimination. This pipeline offers a practical, scalable approach to advancing precision agriculture by enhancing the semantic understanding of crop fields.