The presence of light absorption and scattering in underwater conditions results in underwater images with missing details, low contrast, and color bias. The current deep learning-based methods bring unlimited potential for underwater image restoration (UIR) tasks. These methods, however, do not adequately take into account the inconsistency of the attenuation of different color channels and spatial regions when performing image restoration. To solve these gaps, we propose a dual information modulation network (DIMN) for accurate UIR tasks. To be specific, we design a multi-information enhancement module (MIEM), empowered by spatial-aware attention block (SAAB) and multi-scale structural Transformer block (MSTB), to guide the inductive bias of image degradation processes under nonhomogeneous media distributions. SAAB focuses on different spatial locations, capturing more spatial-aware cues to correct color deviations and recover details. MSTB utilizes the difference and complementarity between features at different scales to effectively complement the network’s structural and global perceptual capabilities, enhancing image sharpness and contrast further. Experimental results reveal that the proposed DIMN exceeds most state-of-the-art UIR methods. Our code and results are available at: https://github.com/wwaannggllii/DIMN.
Underwater images super-resolution (SR) is a challenging task due to underwater images usually contain severely blurred details, color distortion, and low contrast. Although numerous deep learning-based methods have been developed to solve these problems, these methods suffer from huge model parameters and computations. To address this gap, we propose a frequency-domain enhanced attention network (FEAN), supported by a series of frequency-enhanced attention modules (FEAM), for accurate underwater SR. Specifically, we start by utilizing a Gaussian filter to decompose the features into high and low frequencies and pass them to the FEAM. Then, in the high-frequency path, we propose a multi-scale attention enhancement block (MAEB) to extract rich image texture information. While in the low-frequency path, we perform a simple convolutional operation to realize the brightness and contrast adjustment of the image. Further, we devise a channel attention fusion block (CAFB) to integrate the enhanced high and low-frequency features to further strengthen the powerful representational capability of the network. Finally, we employ two convolutions to further modulate the features on the high-frequency path for effective color bias correction and detail enhancement. Experimental results show that our FEAN performs better than other underwater SR methods on the USR-248 dataset, with PSNR values of 29.97 dB, 26.23 dB, and 23.99 dB, corresponding to $\times 2$ , $\times 4$ , and $\times 8$ scale factors.
Multi-modal object re-identification (ReID) is a challenging task that seeks to identify objects across different image modalities by leveraging their complementary information. Traditional CNN-based methods are constrained by limited receptive fields, whereas Transformer-based approaches are hindered by high computational demands and a lack of convolutional biases. To overcome these limitations, we propose a novel fusion framework named MambaReID, integrating the strengths of both architectures with the effective VMamba. Specifically, our MambaReID consists of three components: Three-Stage VMamba (TSV), Dense Mamba (DM), and Consistent VMamba Fusion (CVF). TSV efficiently captures global context information and local details with low computational complexity. DM enhances feature discriminability by fully integrating inter-modality information with shallow and deep features through dense connections. Additionally, with well-aligned multi-modal images, CVF provides more granular modal aggregation, thereby improving feature robustness. The MambaReID framework, with its innovative components, not only achieves superior performance in multi-modal object ReID tasks, but also does so with fewer parameters and lower computational costs. Our proposed MambaReID’s effectiveness is validated by extensive experiments conducted on three multi-modal object ReID benchmarks.
6 February 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
The advent of convolutional neural networks (CNNs) has brought substantial progress in image super-resolution (SR) reconstruction. However, most SR methods pursue deep architectures to boost performance, and the resulting large model sizes are impractical for real-world applications. Furthermore, they insufficiently explore the internal structural information of image features, disadvantaging the restoration of fine texture details. To solve these challenges, we propose a lightweight architecture based on a CNN named attention-directed feature aggregation network (AFAN), consisting of chained stacking multi-aware attention modules (MAAMs) and a simple channel attention module (SCAM), for image SR. Specifically, in each MAAM, we construct a space-aware attention block (SAAB) and a dimension-aware attention block (DAAB) that individually yield unique three-dimensional modulation coefficients to adaptively recalibrate structural information from an asymmetric convolution residual block (ACRB). The synergistic strategy captures multiple content features that are both space-aware and dimension-aware to preserve more fine-grained details. In addition, to further enhance the accuracy and robustness of the network, SCAM is embedded in the last MAAM to highlight channels with high activated values at low computational load. Comprehensive experiments verify that our proposed network attains high qualitative accuracy while employing fewer parameters and moderate computational requirements, exceeding most state-of-the-art lightweight approaches.