Progressive deep image compression (DIC) with hybrid contexts is an under-investigated problem that aims to jointly maximize the utility of a compressed image for multiple contexts or tasks under variable rates. In this paper, we consider the contexts of image reconstruction and classification. We propose a DIC framework, called residual-enhanced mask-based progressive generative coding (RMPGC), designed for explicit control of the performance within the rate-distortion-classification-perception (RDCP) trade-off. Three independent mechanisms are introduced to yield a semantically structured latent representation that can support parameterized control of rate and context adaptation. Experimental results show that the proposed RMPGC outperforms a benchmark DIC scheme using the same generative adversarial nets (GANs) backbone in all six metrics related to classification, distortion, and perception. Moreover, RMPGC is a flexible framework that can be applied to different neural network backbones. Some typical implementations are given and shown to outperform the classic BPG codec and four state-of-the-art DIC schemes in classification and perception metrics, with a slight degradation in distortion metrics. Our proposal of a nonlinear-neural-coded and richly structured latent space makes the proposed DIC scheme well suited for image compression in wireless communications, multi-user broadcasting, and multi-tasking applications.

Deep image coding (DIC) for hybrid application contexts has recently attracted significant research interest because of its potential to support both human and machine visual tasks. Since the regions of interest (ROI) are different for different application contexts, it is important to design an adaptive image coding mechanism in practical DIC. In this paper, we propose the first quantization-based adaptive DIC framework for hybrid contexts of image reconstruction and classification. This framework can be applied to upgrade existing fixed-rate DIC models into adaptive DIC for hybrid contexts. It consists of two key modules: a semantics-based ROI mask generation module, and a module for generating ROI gain and inverse gain matrices. These matrices are used to control the quantization accuracy of different latent vector elements, thereby achieving encoding at different rates while prioritizing the reconstruction quality of the ROI. Moreover, we propose a five-stage training method for the quantization-based adaptive DIC model to optimize the rate-distortion-classification-perception (RDCP) tradeoff. Experiments over a wide rate range show that our method achieves superior RDCP tradeoff performance. Compared to the benchmark scheme BM-CHENG, the proposed algorithm improves the classification accuracy by an average of 15%. The average relative improvements on various metrics, such as natural image quality evaluator (NIQE), learned perceptual image patch similarity (LPIPS), and feature similarity index measure (FSIM) are about 22%, 47%, and 1%, respectively. The proposed algorithm is a promising candidate for fast adaptive coding with low-complexity constraints.

This paper proposes a new algorithm for adaptive deep image compression (DIC) that can compress images for different purposes or contexts at different rates. The algorithm can compress images with semantic awareness, which means classification-related semantic features are better protected in lossy image compression. It builds on the existing conditional encoder-based DIC method and adds two features: a model-based rate-distortion-classification-perception (RDCP) framework to control the trade-off between rate and performance for different contexts, and a mechanism to generate coding conditions based on image complexity and semantic importance. The algorithm outperforms the QMAP2021 benchmark on the ImageNet dataset. Over the tested rate range, it improves the classification accuracy by $11\%$ and the perceptual quality by $12.4\%$, $32\%$, and $1.3\%$ on average for NIQE, LPIPS, and FSIM metrics, respectively.