What does this research mean for the field?

A novel 2D multi-modal Segment Anything Model (mmSAM) outperforms standard single-modal SAM, 3D nnUNet, and thresholding methods for tumor segmentation in multi-tracer oncologic PET/CT images, with fine-tuning significantly enhancing cross-tracer accuracy. Novelty: ClaimNovelty.METHODOLOGICAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

To develop and evaluate a multi-modal segmentation model (mmSAM) for accurate lesion segmentation in PET/CT images using multiple tracers.

May 19, 2026Open Access

Multi-modal segment anything model (mmSAM) for tumor segmentation in multi-tracer oncologic PET/CT

Key Points

To develop and evaluate a multi-modal segmentation model (mmSAM) for accurate lesion segmentation in PET/CT images using multiple tracers.
Utilized the AutoPET 2024 dataset with 369 subjects for 18F-PSMA, and 170 subjects each for 18F-FDG and 68Ga-PSMA PET/CT images.
Trained mmSAM on the primary dataset, divided into training, validation, and testing sets, and evaluated its transferability on two additional datasets.
Compared mmSAM's performance to standard single modal SAM, 3D nnUNet, and thresholding-based methods using various metrics.
mmSAM achieved superior metrics on the primary 18F-PSMA dataset with mean Dice of 0.76 and TPR of 100%.
Fine-tuning improved its performance on cross-tracer datasets, achieving a mean Dice of 0.81 for 68Ga-PSMA.
Statistical significance was established using the Wilcoxon test, with all comparisons showing p < 0.05.

Abstract

Lesion segmentation is important in targeted radionuclide therapy dosimetry. In this study, we proposed a novel 2D multi-modal Segment Anything Model (mmSAM) for lesion segmentation in whole-body multi-tracer PET/CT images. The AutoPET 2024 dataset, including 18F-PSMA (369 subjects), 18F-FDG (170 subjects) and 68Ga-PSMA (170 subjects) PET/CT images and their corresponding tumor masks, was used in this study. The 18F-PSMA PET/CT dataset was used as the primary dataset and was divided into 233: 36: 100 for training, validation and testing the mmSAM, using both PET and CT input. The transferability of the primary model was evaluated on 100 patients on other 2 datasets, without and with fine-tuning using 70 cases of 18F-FDG and 68Ga-PSMA respectively. Standard single modal 2D SAM with only PET input, 3D nnUNet with two-channel PET/CT input and the thresholding-based method were also implemented for comparison. Mean Dice, the 95th percentile Hausdorff distance (mean HD95), mean standardized uptake value (mean |SUVmean| error), metabolic tumor volume (mean |MTV| error), true positive rate (TPR), positive predictive value (PPV), and false discovery rate (FDR) were computed for all tumors. Statistical significance among different segmentation methods was evaluated using the Wilcoxon test. mmSAM achieved the best performance as compared to other segmentation methods for the primary 18F-PSMA dataset (mean Dice/HD95/|SUVmean| error/|MTV| error/TPR/PPV/FDR = 0.76/1.92 mm/5.10%/14.60%/100%/96.51%/3.49%, all p < 0.05), without fine-tuning (mean Dice/HD95/|SUVmean| error/|MTV| error/TPR/PPV/FDR = 0.61/2.28 mm/ 15.83%/27.71%/100%/78.91%/21.09% for 18F-FDG; 0.77/1.33 mm/5.55%/17.96%/100% /97.78%/2.22% for 68Ga-PSMA, p < 0.05), as well as with fine-tuning (mean Dice/HD95/|SUVmean| error/|MTV| error/TPR/PPV/FDR = 0.65/1.81 mm/6.62%/13.93%/100%/ 80.50%/19.50% for 18F-FDG; 0.81/1.15 mm/4.62%/14.30%/100%/99.30%/0.70% for 68Ga-PSMA, p < 0.05) on the cross-tracer datasets. The proposed mmSAM is promising for lesion segmentation in multi-tracer oncologic PET/CT images. Fine-tuning significantly enhances segmentation accuracy for cross-tracer studies.

AI에게 질문

Bookmark

View Full Paper