Multimodal Deep Learning Model for Real-Time Severity Assessment of Maize Stalk Rot Integrating Visual and Soil Sensing Data

• Multimodal model fuses RGB imagery with in-field soil moisture sensing • Late-fusion fuzzy logic improves robustness under heterogeneous field conditions • Achieves 96.9% accuracy and outperforms vision-only methods by up to 5.3% • Real-time severity assessment deployed on a mobile robotic platform • Provide a scalable approach for reliable disease monitoring in maize fields Maize stalk rot is a destructive fungal disease that reduces yield by weakening stalks and causing lodging. However, current methods for detecting the disease rely solely on artificial vision, which limits comprehensive assessment of severity by omitting critical environmental variables such as soil moisture. This study proposes a multimodal deep learning model for real-time assessment of maize stalk rot severity, integrating visual features and soil moisture data collected by a mobile robot. An enhanced YOLOv8n was used to localize stalk regions, followed by a VGG16 classifier that assigns three severity levels: healthy, moderately infected, and severely infected. Simultaneously, volumetric water content (VWC) near the root zone was measured using moisture sensors, and temporal dynamics were analyzed to obtain severity thresholds from soil moisture patterns. A fuzzy logic-based late fusion integrates soil and image predictions, improving robustness under heterogeneous field conditions. Field validation on the test subset showed that the multimodal model achieved 94.4%, 94.1%, and 94.1% in overall precision, recall, and F1-score, respectively, outperforming the vision-only baseline by up to 7.8% in accuracy. These results indicate that the proposed model provides a reliable and scalable assessment of stalk rot severity under real agricultural conditions