Automatic Crop Identification and Growth Stage Determination of Staple Crops Using Farmland Images and Anomaly Detection Strategy

• Anomaly detection was applied to filter unknown or low-quality images • A ViT-based model was applied to identify 53 types of crops (94.83% acc) • Two ViT-Small models were used to classify rice and maize growth stages • System processes 2.2 million smartphone images with 97% automation • Modular pipeline enables scalable and near real-time agricultural monitoring Crop production serves as the foundation of global food security. Crop yields need to be estimated for the purposes of food price stabilization. Conventionally, crop yields were estimated through agricultural field surveys, in which investigators visited fields and recorded crop types and their growth stages. The manual survey approach, however, is time-consuming and demands extensive professional training. Moreover, farmland images frequently contain anomaly images, such as unknown crops, non-crop elements, and unqualified images. Feeding these anomaly images into a classifier can cause misclassification. This study proposes a fully automated pipeline that (i) filters out anomaly images, (ii) classifies 53 crop types, and (iii) identifies the growth stages of rice and maize in farmland images acquired by smartphones. The proposed pipeline consists of five components: (a) a target crop feature extractor (TCFE), (b) a target crop detection model (TCDM), (c) a target crop classifier (TCC), (d) growth stage classifiers (GSCs) for staple crops (rice and maize in Taiwan), and (e) an unknown-crop detector (UCD). First, a Vision Transformer-Base (ViT-Base) model was trained to identify the 53 target crop types. The feature extractor of the trained ViT-Base model (i.e., with the final classification layer removed) was used as the TCFE. Subsequently, a support vector machine (SVM) was trained as the TCDM to filter out anomaly images (images that do not fall into the 53 crop types) using the features from the TCFE. On the other hand, the complete ViT-Base model was used as the TCC. Furthermore, a SVM was trained as the UCD to determine if an anomaly image rejected by TCDM contains unknown crops that require manual classification. Two ViT-Small (ViT-Small) models were trained as GSCs to identify the growth stages of rice and maize. The proposed pipeline was tested using 2.2 million smartphone-acquired farmland images. The TCDM, TCC, rice GSC, and maize GSC, respectively, achieved accuracies of 95.77%, 94.83%, 96.63%, and 95.09%. The proposed approach reduces manual effort by approximately 97%. The pipeline offers a scalable and real-time solution for modern agricultural surveys.