Natural images are often degraded by complex, composite degradations such as rain, snow, and haze, which adversely impact downstream vision applications. While existing image restoration efforts have achieved notable success, they are still hindered by two critical challenges: limited generalization across dynamically varying degradation scenarios and a suboptimal balance between preserving local details and modeling global dependencies. To overcome these challenges, we propose M2Restore, a novel Mixture-of-Experts (MoE)-based Mamba-CNN fusion framework for efficient and robust all-in-one image restoration. M2Restore introduces three key contributions: First, to boost the model's generalization across diverse degradation conditions, we exploit a CLIP-guided MoE gating mechanism that fuses task-conditioned prompts with CLIP-derived semantic priors. This mechanism is further refined via cross-modal feature calibration, which enables precise expert selection for various degradation types. Second, to jointly capture global contextual dependencies and fine-grained local details, we design a dual-stream architecture that integrates the localized representational strength of CNNs with the long-range modeling efficiency of Mamba. This integration enables collaborative optimization of global semantic relationships and local structural fidelity, preserving global coherence while enhancing detail restoration. Third, we introduce an edge-aware dynamic gating mechanism that adaptively balances global modeling and local enhancement by reallocating computational attention to degradation-sensitive regions. This targeted focus leads to more efficient and precise restoration. Extensive experiments across multiple image restoration benchmarks validate the superiority of M2Restore in both visual quality and quantitative performance.
20 November 2023 in IEEE Transactions on Industrial Informatics
Haze severely affects the reliability of vision-based industrial systems, and most of the current dehazing methods are not applicable to images captured by ultra-high-definition (UHD) industrial cameras. In this article, we propose a novel dimensional transformation mixer (DMixer) model for recovering haze-free images from UHD haze images. In DMixer, the dimensional transformation module encodes the complete image in multiple stages from different perspectives and associates features from different views by permuting the tensor for efficient long-range dependency modeling. In this way, the global perception capabilities of DMixer are complemented, allowing the quality of the reconstructed UHD images to be improved. Furthermore, DMixer employs a dual-stream network framework that combines local and multiscale features, allowing DMixer to better trade off the performance and efficiency for real industrial systems. Our proposed method enables the real-time processing of UHD images ( $\sim$ 59 fps). Extensive results show that the proposed model outperforms current state-of-the-art methods, with PSNR improvements of 1.94 and 0.95 dB on the 4KID and O-HAZE datasets, respectively.
In the realm of image super-resolution, learningbased methods have made significant progress. However, limited computational resources still restrict their application. This prompts us to develop an efficient method for achieving effective image super-resolution. In this paper, we propose a novel adaptive feature selection modulation network (AFSMNet) tailored for efficient image super-resolution. Specifically, we design feature modulation blocks, which include the adaptive feature selection modulation (AFSM) module and the self-gating feed-forward network (SFN). The AFSM module dynamically computes the importance of each feature channel. For channels with differing levels of importance, we employ distinct processing strategies, thereby concentrating the computational resources of the network on the more critical features as much as possible. This approach facilitates the maintenance of a low computational cost without compromising performance. The SFN restricts the flow of irrelevant feature information within the network through a simple gating mechanism. In this way, our method achieves efficient and effective image super-resolution. Extensive experiment results show that the proposed method achieves a better trade-off between reconstruction performance and computational efficiency compared to the current state-of-the-art lightweight super-resolution methods
16 September 2024 in IEEE Signal Processing Letters
Recently, large models (Segment Anything model) came on the scene to provide a new baseline for polyp segmentation tasks. This demonstrates that large models with a sufficient image level prior can achieve promising performance on a given task. In this paper, we unfold a new perspective on polyp segmentation modeling by leveraging the Depth Anything Model (DAM) to provide depth prior to polyp segmentation models. Specifically, the input polyp image is first passed through a frozen DAM to generate a depth map. The depth map and the input polyp images are then concatenated and fed into a convolutional neural network with multiscale to generate segmented images. Extensive experimental results demonstrate the effectiveness of our method, and in addition, we observe that our method still performs well on images of polyps with noise.