As the role of remote sensing continues to grow, the fusion technology of low-spatial-resolution hyperspectral images and high-spatial-resolution multispectral images has become increasingly critical. Traditional methods rely on fixed rules and exhibit poor robustness, whereas deep learning methods struggle to establish efficient interactions between local and global information due to the complexity of their underlying networks. Therefore, we propose a deep learning fusion module that combines pixel-wise adaptive wavelet transform with a spectral–spatial dual-branch extraction. Firstly, by utilizing the unique properties of the wavelet transform, it is possible to effectively preserve spectral information and extract spatial edge features, thereby achieving preliminary fusion by leveraging both low-frequency and high-frequency components. To compensate for the lack of nonlinear expression capability in the wavelet transform, a dual-branch parallel extraction of spectral and spatial features is subsequently performed in the deep learning module. The Multi-Scale Group Convolution module (MSGC) is utilized to extract spectral information, while the Spectral Compression and Spatially Guided Gating Module (SCSGM) is employed to extract spatial information, thereby enhancing the data’s adaptive capability. A bidirectional attention mechanism is interspersed within the module to capture complementary information across different scales, ultimately reconstructing a high-resolution hyperspectral image. Finally, the proposed fusion strategy demonstrates superior performance in practical image reconstruction, outperforming more than ten state-of-the-art fusion methods.