Deep hierarchical multiscale attention networks for image super-resolution

With the rapid development of deep learning, the task of image super-resolution has made significant progress. However, as model depth increases, training becomes more difficult and fails to capture coarse-grained and fine-grained information simultaneously. To solve these issues, we propose Deep Hierarchical Multiscale Attention Networks (DHMA). First, we use a residual nested residual structure to improve the propagation of gradient information to address the challenge of training deep networks, which consists of multiple residual groups, each composed of multiple residual blocks. In addition, we propose a Hierarchical Multiscale Attention (HMA) module to capture both coarse-grained and fine-grained features in deep networks. This method imitates how humans observe things, first focusing on salient objects and then observing details. Specifically, we first segment the feature map horizontally into several parts and then perform Global Aware Attention (GAA) learning on each part. Where GAA can learn global structural information. Next, we use the Adaptive Feature Fusion (AFF) module to fuse the learned information of each part into the current layer's features. Finally, we stack the features of each layer to construct a hierarchical multiscale structure and obtain features of different scales. HMA is a lightweight plug-and-play module that can be applied to existing models. Extensive experiments demonstrate the effectiveness and outstanding performance of the DHMA.