AKSeg: combining variable kernel convolution with Nash multi task learning

Abstract In complex image multi-task learning, the precision and effectiveness of feature extraction are often constrained by traditional convolutional methods, including issues with scale invariance and difficulties in feature sharing across tasks. To tackle these challenges, we introduce a straightforward but efficient model called AKSeg. AKSeg effectively combines Adaptive Kernel Convolution (AKConv) with a Nash-based multi-task learning strategy within the SegNet framework, improving the model’s ability to capture multi-task features from images. First, AKSeg incorporates AKConv into the encoder layer of the SegNet. By dynamically adjusting the size and shape of convolutional kernels, AKSeg can accurately capture features of various scales from images. Then, a Nash game strategy is employed in the AKSeg to update parameters and weights as tasks change. To tackle challenges like feature extraction across varying tasks and improve multi-task learning capability, AKSeg utilizes multiple channels to efficiently extract distinct features for each task. Additionally, AKSeg addresses the issue of dead Rectified Linear Unit (ReLU) by eliminating inactive neurons through the use of Leaky Rectified Linear Unit (LeakyReLU). Extensive experimental results demonstrate that the AKSeg outperforms baseline methods remarkably. Specifically, AKSeg reduces the loss value by 4.48%, improves mIoU by 4.52%, and increases pixel accuracy by 1.73%. These results validate the usefulness of AKSeg in enhancing image multi-task learning performance.