7 August 2023 in IEEE Transactions on Circuits and Systems for Video Technology
The proliferation of deep learning-based machine vision applications has given rise to a new type of compression, so called video coding for machine (VCM). VCM differs from traditional video coding in that it is optimized for machine vision performance instead of human visual quality. In the feature compression track of MPEG-VCM, multi-scale features extracted from images are subject to compression. Recent feature compression works have demonstrated that the versatile video coding (VVC) standard-based approach can achieve a BD-rate reduction of up to 96% against MPEG-VCM feature anchor. However, it is still sub-optimal as VVC was not designed for extracted features but for natural images. Moreover, the high encoding complexity of VVC makes it difficult to design a lightweight encoder without sacrificing performance. To address these challenges, we propose a novel multi-scale feature compression method that enables both the end-to-end optimization on the extracted features and the design of lightweight encoders. The proposed model combines a learnable compressor with a multi-scale feature fusion network so that the redundancy in the multi-scale features is effectively removed. Instead of simply cascading the fusion network and the compression network, we integrate the fusion and encoding processes in an interleaved way. Our model first encodes a larger-scale feature to obtain a latent representation and then fuses the latent with a smaller-scale feature. This process is successively performed until the smallest-scale feature is fused and then the encoded latent at the final stage is entropy-coded for transmission. The results show that our model outperforms previous approaches by at least 52% BD-rate reduction and has ×5 to ×27 times less encoding time for object detection. It is noteworthy that our model can attain near-lossless task performance with only 0.002-0.003% of the uncompressed feature data size.
22 August 2023 in IEEE Access
Machine vision-based intelligent applications that analyze video data collected by machines are rapidly increasing. Therefore, it is essential to efficiently compress a large volume of video data for machine consumption. Accordingly, the Moving Picture Experts Group (MPEG) has been developing a new video coding standard called Video Coding for Machines (VCM), aimed at video consumed by machines rather than humans. Recently, studies have demonstrated that multi-scale feature compression (MSFC)-based feature compression methods significantly improve the performance of MPEG-VCM. This paper proposes an efficient MSFC (eMSFC) method with quantization parameter (QP)-adaptive feature channel truncation. The proposed eMSFC incorporates an MSFC network with a selective learning strategy (SLS) and Versatile Video Coding (VVC)-based compression. The SLS extracts a single-scale feature from the input image, arranged in order of channel-wise importance. The size of the single-scale feature is adaptively adjusted by truncating the feature channels according to the QP. The truncated feature is efficiently compressed using VVC. Compared to the VCM feature anchor, the experimental results reveal that the proposed method provides a 98.72%, 98.34%, and 98.04% Bjontegaard delta rate gain for machine vision tasks of instance segmentation, object detection, and object tracking, respectively. The proposed method performed best among the “Call for Evidence” response technologies in MPEG-VCM.
27 April 2015 in IEEE Transactions on Broadcasting
In this paper, we demonstrate the structure and characteristics of the bandwidth-efficient stereoscopic 3-D broadcasting system entitled service compatible 3-D-TV using main and mobile hybrid delivery (SC-MMH), which enables the broadcasters to provide three different types of services (2-D fixed, mobile, and 3-D) by transmitting only the two legacy broadcast streams (2-D fixed and mobile). By combining the two existing broadcast networks, the proposed system provides high quality 3-D services, fully guarantees backward-compatibility with the legacy fixed/mobile 2-D receivers, and maximizes the channel efficiency. One major feature of the proposed system is bit-rate saving based on the video resolution asymmetry between the left and right eye views, also referred to as “mixed-resolution.” Although it is well-known wisdom that the mixed-resolution stereoscopic videos are perceived close to the higher resolution based on the binocular rivalry suppression theory, the perceived quality also depends on the factors such as scale ratio, viewing distance, bit-rate distribution, and types of codecs. Therefore, we conducted the objective and subjective tests to measure the service quality of the proposed system and find the optimum bit-rate distribution points, and with the results, we demonstrate that SC-MMH can provide the satisfactory quality when the proper level of bit-rate is allocated to the mobile service.
12 January 2026 in ACM Transactions on Multimedia Computing, Communications, and Applications
Research on entropy model-based Learned Image Compression (LIC) has been actively progressing, leading to rapid advancements in coding efficiency. Beyond improvements in coding efficiency, LIC methods have also been explored for practical codec development. Despite these advancements, research on learned Progressive Image Coding (PIC) remains in its early stages. PIC aims to encode multiple quality levels into a single bitstream, improving bitstream versatility and achieving higher compression efficiency than simulcast compression. Existing learned PIC methods hierarchically quantize transformed latent representations with varying quantization step sizes. More specifically, these approaches progressively compress the additional information needed for quality improvement, considering that a wider quantization interval for lower-quality compression includes multiple narrower subintervals for higher-quality compression. However, they rely on handcrafted quantization hierarchies, leading to suboptimal compression efficiency. In this article, we propose a learned PIC method that first exploits learned quantization step sizes for each quantization layer. We also incorporate selective compression, ensuring that only essential representation components are retained in each quantization layer. Our experimental results demonstrate that the proposed method significantly enhances coding efficiency compared to the existing approaches while also reducing decoding time and model size. The source code is publicly available at https://github.com/JooyoungLeeETRI/DeepHQ .
13 August 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Video coding for machines (VCM) is an emerging approach in video compression designed to optimize content for machine analysis tasks. Although VCM was initially developed for machine vision, scalable coding frameworks have been developed to support both machine-driven analysis and human viewing as required. In this work, we focus on scenarios where high-quality encoding of regions of interest (ROIs) for machine vision and low-bitrate encoding of the background (BG) for human vision. At the decoder, severely degraded BG quality in reconstructed frames makes them unsuitable for viewing; therefore, restoring the degraded BGs by leveraging high-quality ROIs is essential. To this end, we propose the Gradient-Guided Diffusion Restoration (GGDR) algorithm, which integrates a pretrained generative diffusion model with content-aware supervision and adaptive refinement mechanisms to restore severely degraded regions robustly while maintaining visual consistency across the entire frame. The GGDR algorithm consists of two key components: (i) a content-aware supervision mechanism that preserves salient features and structural information in the input image, ensuring superior performance even with challenging high-variance inputs and (ii) a refinement block that guides the generation process of the pretrained diffusion model based on a degradation model and structural guidance. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art algorithms both qualitatively and quantitatively.