Hyperparameter Optimization of TransUNet: A Comparative Study on Learning Rates and Optimizers for Medical Image Segmentation

Medical image segmentation is widely applied these days for medical diagnosis. Particularly the model, TransUNet with the convolutional neural network (CNN) hybridized Transformer as the encoder shows the prominence in the multi-organ segmentation in the abdominal Synapse dataset, which is maintained as the u-shaped architecture overall. However, TransUNet lacks analysis on the choice of the hyperparameters, which only presents the comparison of the effects in terms of other models. This paper researches the training and testing performances on the various learning rates and different optimizers based on the model TransUNet. With the 12 batch size on the training process, this paper compares the training loss and some evaluations for testing through various learning rates and the different optimizers. The comparison method found that the SGD optimizer with the learning rate 0.01 performed better on training process while the testing evaluations are better conveyed through the AdamW optimizer at 0.001 learning rate.