Multi-source hierarchical deep learning framework for fine-grained tea classification using hyperspectral imaging

• Multi-source HSI framework integrates data, feature, and model fusion for classification • Spectral fusion boosts accuracy for fine-grained tea germplasm classification • Hybrid1D2D+Transformer and 2DCNNViT improve classification at fine and image levels • Hybrid1D2D+Transformer reaches 96.87%, 2DCNNViT achieves 98.95% and 96.88% on VI/NI • Hybrid models extract multi-scale features through local and global context learning Accurate classification of tea resources is crucial for quality control and variety protection. Traditional methods rely on manual expertise or destructive testing, making non-destructive, efficient identification challenging. Hyperspectral imaging (HSI) offers a promising alternative, but many existing HSI-based methods focus solely on spectral features, overlooking spatial heterogeneity and operating at a single scale, which limits performance in fine-grained classification tasks. Additionally, raw hyperspectral data are often underutilized, leading to suboptimal spatial structure exploitation. This study proposed a multi-source, data-driven hierarchical classification framework using HSI data from the visible near-infrared (VI) and near-infrared (NI) bands. Average spectra, pixel-level spectra, and hyperspectral images represent different data levels. Among traditional classifiers applied to average spectra, the highest accuracy achieved was 72.22%. For pixel-spectra classification, two deep learning models—Hybrid1D2D and Hybrid1D2D+Transformer—were developed. Hybrid1D2D captures multi-scale spectral and spatial features, improving accuracy by at least 14% over 1DCNN and 2DCNN, based on the VIpix+NIpix dataset, which provided the best results after data fusion. Building on this, Hybrid1D2D+Transformer improved fine-grained tea classification, achieving 96.87±0.22% accuracy by capturing long-range dependencies. For hyperspectral image classification, the proposed 2DCNNViT model combines convolutional networks for local feature extraction with Transformer architectures for global context modeling, achieving 98.95±0.05% and 96.88±0.12% accuracy on the VI and NI datasets, respectively. The proposed models demonstrated strong robustness and generalization across datasets, highlighting the complementary advantages of convolutional and Transformer-based representations. This study confirms the potential of hyperspectral imaging coupled with deep learning fusion strategies for intelligent, non-destructive, and high-precision identification of tea varieties, offering a valuable tool for improving quality assessment and resource management in crop production.