Compressive Sensing (CS) has emerged as a transformative technique in image compression, offering innovative solutions to challenges in efficient signal representation and acquisition. This paper provides a comprehensive exploration of the key components within the domain of CS applied to image and video compression. We delve into the fundamental principles of CS, highlighting its ability to efficiently capture and represent sparse signals. The sampling strategies employed in image compression applications are examined, emphasizing the role of CS in optimizing the acquisition of visual data. The measurement coding techniques leveraging the sparsity of signals are discussed, showcasing their impact on reducing data redundancy and storage requirements. Reconstruction algorithms play a pivotal role in CS, and this article reviews state-of-the-art methods, ensuring a high-fidelity reconstruction of visual information. Additionally, we explore the intricate optimization between the CS encoder and decoder, shedding light on advancements that enhance the efficiency and performance of compression techniques in different scenarios. Through a comprehensive analysis of these components, this review aims to provide a holistic understanding of the applications, challenges, and potential optimizations in employing CS for image and video compression tasks.
20 May 2021 in IEEE Transactions on Industrial Informatics
Artificial intelligence of things has brought artificial intelligence to the cutting-edge Internet of Things. In recent years, compressive sensing (CS), which relies on sparsity, is widely embedded and expected to bring more energy efficiency and a longer battery lifetime to IoT devices. Different from the other image compression standards, CS can get various reconstructed images by applying different reconstruction algorithms on coded data. Using this property, it is the first time to propose a deep learning based compressive sensing image enhancement framework using multiple reconstructed signals (CSIE-M). In this article, first, images are reconstructed by different CS reconstruction algorithms. Second, reconstructed images are assessed and sorted by a no-reference quality assessment module before being input to the quality enhancement module by order of quality scores. Finally, a multiple-input recurrent dense residual network is designed for exploiting and enriching the useful information from the reconstructed images. Experimental results show that CSIE-M obtains 1.88–8.07 dB peek-signal-to-noise (PSNR) improvement while the state-of-the-art works achieve a 1.69–6.69 dB PSNR improvement under sampling rates from 0.125 to 0.75. On the other hand, using multiple reconstructed versions of the signal can improve 0.19–0.23 dB PSNR, and only 4% reconstructing time is increasing compared to using a reconstructed signal.
26 October 2023 in IEEE Transactions on Multimedia
Compressive Sensing (CS) surpasses the limitations of the sampling theorem by reducing signal dimensions during sampling. Recent works integrate measurement coding into CS to enhance the compression ratio. However, these works significantly decrease image quality, and both encoding and decoding become time-consuming. This paper proposes a Compressive Sensing based Image Codec with Partial Pre-calculation (CSCP) to solve these issues. The CSCP separates the original reconstruction procedure into two parts: reconstructing the frequency domain data and the inverse calculation. Depending on the feature of the chosen deterministic sensing matrix, the complex reconstruction procedure is reduced to twice matrix-based multiplications, resulting in a low time cost. Moreover, we can further optimize the reconstruction process by moving the frequency domain data reconstruction to the encoder, referred to as the partial pre-calculation process. Then compressing the sparse data in the frequency domain. This approach has two main benefits: 1) it reduces the complexity of the decoder, and 2) it results in less degradation in quality compared to existing measurement coding methods. Additionally, this work proposes the One-Row-Two-Tables strategy for defining Huffman Coding units. This approach leverages the quantized data distribution to improve compression efficiency while maintaining low complexity. In the decoder, the sequence of operations includes Huffman decoding, dequantization, and inverse calculation. Compared to the state-of-the-art, this work decreases 22.61 $\%$ bpp with 17.72 $\%$ increased quality. Meanwhile, time speeds up to 649.13× on the encoder, 11.03× on the decoder, and 288.46× in total.
The existing video compressive sensing (CS) techniques with fixed sampling rates can deliver satisfactory reconstructed quality but necessitate large transmission bandwidth. To overcome this challenge, region-of-interest (ROI) based CS algorithms have been introduced to allocate different coding resources between ROI and non-ROI segments. However, neglecting non-ROI excessively in these algorithms leads to dissatisfied average quality for the eventual reconstruction. In this paper, we integrate the ideas of above methods and propose a novel adaptive video compressive sensing using a low-complexity ROI detection (aVCSR) method in compressed domain. The ROI is detected and sampled by calculating the measurement variance between the reference frame and the subsequent frames. Conversely, the non-ROI is not transmitted but will be reconstructed by utilizing the reference frame through the corresponding position information. In addition, we present a compact method for adapting the threshold value, which allows each frame of a video to have a unique threshold rather than an artificially predetermined fixed value. Moreover, a reference frame updating strategy is developed to improve the versatility of the entire framework. Compared to the state-of-the-art counterparts, extensive experimental results have demonstrated that our proposed methods achieve superior performance while tackling diverse scenes and using a lower sampling rate.
Recently, deep learning-inferred stain deconvolution/separation-based nuclei segmentation works demonstrated significant results by translating low-cost and prevalent immunohistochemical (IHC) slides to more expensive-yet-informative multiplex immunofluorescence (mpIF) images. However, when the input stain style is changed to Hematoxylin and Eosin stain (H&E), which is one of the principal tissue stains used in histology, the stain deconvolution/separation based works can not achieve satisfactory performance because the features of the input stain image are greatly changed. To solve this problem, we integrate stain transfer (H&E->IHC) before stain deconvolution (IHC->mpIF) to revise the image style. Moreover, a new semi-supervised learning strategy collaborating supervised and unsupervised learning processes are employed to diversify training data content for stain deconvolution.Firstly, in the supervised learning process, generative adversarial network (GAN) based image-to-image mapping (stain deconvolution) and inverse mapping models are trained on the dataset of co-registered IHC staining and mpIF staining of the same slides to respectively convert IHC to mpIF (I2m), and mpIF to IHC (m2I). After that, in the unsupervised learning process, high-quality IHC images from m2I model are selected according to the Discriminator score on another unpaired dataset, and then, IHC images are paired with input mpIF as the unsupervised training data for further improving the I2m model. This semi-supervised scheme balances two supervised and unsupervised errors while optimizing to limit the effect of imperfect pseudo inputs but still enhance stain deconvolution. Furthermore, image enhancement is applied after the stain deconvolution model to obtain high-quality segmentation masks. We thoroughly evaluate our method on publicly available benchmark datasets. The results show the proposed model obtains significant improvement compared to the SOTAs.