Well-known compressed sensing (CS) is widely used in image acquisition and reconstruction. However, accurately reconstructing images from measurements at low sampling rates remains a considerable challenge. In this paper, we propose a novel Transformer-based hybrid architecture (dubbed TransCS) to achieve high-quality image CS. In the sampling module, TransCS adopts a trainable sensing matrix strategy that gains better image reconstruction by learning the structural information from the training images. In the reconstruction module, inspired by the powerful long-distance dependence modelling capacity of the Transformer, a customized iterative shrinkage-thresholding algorithm (ISTA)-based Transformer backbone that iteratively works with gradient descent and soft threshold operation is designed to model the global dependency among image subblocks. Moreover, the auxiliary convolutional neural network (CNN) is introduced to capture the local features of images. Therefore, the proposed hybrid architecture that integrates the customized ISTA-based Transformer backbone with CNN can gain high-performance reconstruction for image compressed sensing. The experimental results demonstrate that our proposed TransCS obtains superior reconstruction quality and noise robustness on several public benchmark datasets compared with other state-of-the-art methods. Our code is available on TransCS.

The convolutional neural network (CNN)-based reconstruction methods have dominated the compressive sensing (CS) in recent years. However, existing CNN-based approaches show potential restrictions in capturing non-local similarity of images, because of the intrinsic characteristic of convolutional layers, $\mathit{i.e.}$ , locality and weight sharing. In parallel, the emerging Transformer architecture shows fine capacity in modeling long-distance correlations onto embedded tokens for language and images. Yet vanilla Transformer does not exceed CNN-based networks considerably but shows roughly comparable performance, and the culprit can be the missing of sophisticated inductive bias regarding the local image structures. In this article, to eliminate the restrictions of the aforementioned paradigms, we propose a Transformer-based hierarchical framework, dubbed TCS-Net, for compressive image sensing (or image compressive sensing) with a $\mathit{patch-to-pixel}$ manner. Concretely, the proposed TCS-Net consists of an image acquisition module and a reconstruction module (includes two key decoding phases: a patch-wise decoding phase and a pixel-wise decoding phase). The acquisition module can implement data-driven image sampling by jointly learning with the decoding phases. By adjusting the Transformer architecture to the $\mathit{patch-to-pixel}$ multi-stage pattern, our reconstruction module can gradually decode the CS measurements from the patch-wise outlines to the pixel-wise textures, thereby building a high-precision mapping for image reconstruction. Extensive experiments on several datasets verify that the proposed TCS-Net outperforms existing state-of-the-art image CS methods by considerable margins.

Deep unrolling architectures have revitalized compressive sensing (CS) by seamlessly blending deep neural networks with traditional optimization-based reconstruction algorithms. In pursuit of an efficient and deep interpretable approach, we propose LTwIST for CS problem, a novel deep unrolling framework that draws inspiration from the well-known two-step iterative shrinkage thresholding (TwIST) algorithm. LTwIST uses a trainable sensing matrix to adaptively learn structural information in images, and introduces a customized U-block architecture to solve the proximal mapping of nonlinear transformations connected with the sparsity-inducing regularizer. Specifically, each iteration recovery step of LTwIST corresponds to an iterative update step of the traditional TwIST algorithm. Moreover, the proposed method is designed to learn all the parameters end-to-end without manual tuning such as shrinkable thresholds, step sizes, etc. As a result, LTwIST obviates the need for manual parameter optimization, allows for high-quality image recovery and provides unambiguous interpretability. Moreover, our proposed LTwIST is also applicable to CS-based magnetic resonance imaging and exhibits a strong reconstruction performance. Extensive experiments on several public benchmark datasets demonstrate that the proposed LTwIST outperforms existing state-of-the-art deep CS methods by considerable margins in terms of quality evaluation metrics and visual performance. Our code is available on LTwIST.

Block compressive sensing (CS) is a well-known signal acquisition and reconstruction paradigm with widespread application prospects in science, engineering, and cybernetic systems. However, state-of-the-art block-based image CS (BCS) methods generally suffer from two issues. The sparsifying domain and the sensing matrices widely used for image acquisition are not data driven and, thus, both the features of the image and the relationships among subblock images are ignored. Moreover, it requires to address a high-dimensional optimization problem with extensive computational complexity for image reconstruction. In this article, we provide a deep learning (DL) strategy for BCS, called AutoBCS, which automatically takes the prior knowledge of images into account in the acquisition step and establishes a reconstruction model for performing fast image reconstruction. More precisely, we present a learning-based sensing matrix to accomplish image acquisition, thereby capturing and preserving more image characteristics than those captured by the existing methods. In addition, we build a noniterative reconstruction network, which provides an end-to-end BCS reconstruction framework to maximize image reconstruction efficiency. Furthermore, we investigate comprehensive comparison studies with both traditional BCS approaches and newly developed DL methods. Compared with these approaches, our proposed AutoBCS can not only provide superior performance in terms of image quality metrics (SSIM and PSNR) and visual perception but also automatically benefit reconstruction speed.

Deep compressive sensing (CS) has become a prevalent technique for image acquisition and reconstruction. However, existing deep learning (DL)-based CS methods often encounter challenges such as block artifacts and information loss during iterative reconstruction, particularly at low sampling rates, resulting in a reduction of reconstructed details. To address these issues, we propose NesTD-Net, an unfolding-based architecture inspired by the NESTA algorithm, designed for image CS. NesTD-Net integrates DL modules into NESTA iterations, forming a deep network that continuously iterates to minimize the ℓ ₁ -norm CS problem, ensuring high-quality image CS. Utilizing a learned sampling matrix for measurements and an initialization module for initial estimate, NesTD-Net then introduces Iteration Sub-Modules derived from the NESTA algorithm ( i.e ., Y _k , Z _k , and X _k ) during reconstruction stages to iteratively solve the ℓ ₁ -norm CS reconstruction. Additionally, NesTD-Net incorporates a Dual-Path Deblocking Structure (DPDS) to facilitate feature information flow and mitigate block artifacts, enhancing image detail reconstruction. Furthermore, DPDS exhibits remarkable versatility and demonstrates seamless integration with other unfolding-based methods, offering the potential to enhance their performance in image reconstruction. Experimental results demonstrate that our proposed NesTD-Net achieves better performance compared to other state-of-the-art methods in terms of image quality metrics such as SSIM and PSNR, as well as visual perception on several public benchmark datasets. Our code is available at NesTD-Net.

Image compressed sensing (ICS) has been extensively applied in various imaging domains due to its capability to sample and reconstruct images at subNyquist sampling rates. The current predominant approaches in ICS, specifically pure convolutional networks (ConvNets)-based ICS methods, have demonstrated their effectiveness in capturing local features for image recovery. Simultaneously, the Transformer architecture has gained significant attention due to its capability to model global correlations among image features. Motivated by these insights, we propose a novel hybrid network for ICS, named MTC-CSNet, which effectively combines the strengths of both ConvNets and Transformer architectures in capturing local and global image features to achieve high-quality image recovery. Particularly, MTC-CSNet is a dual-path framework that consists of a ConvNets-based recovery branch and a Transformer-based recovery branch. Along the ConvNets-based recovery branch, we design a lightweight scheme to capture the local features in natural images. Meanwhile, we implement a Transformer-based recovery branch to iteratively model the global dependencies among image patches. Ultimately, the ConvNets-based and Transformer-based recovery branches collaborate through a bridging unit, facilitating the adaptive transmission and fusion of informative features for image reconstruction. Extensive experimental results demonstrate that our proposed MTC-CSNet surpasses the state-of-the-art methods on various public datasets. The code and models are publicly available at MTC-CSNet.