Invertible image decolorization is a useful color compression technique to reduce the cost in multimedia systems. Invertible decolorization aims to synthesize faithful grayscales from color images, which can be fully restored to the original color version. In this paper, we propose a novel color compression method to produce invertible grayscale images using invertible neural networks (INNs). Our key idea is to separate the color information from color images, and encode the color information into a set of Gaussian distributed latent variables via INNs. By this means, we force the color information lost in grayscale generation to be independent of the input color image. Therefore, the original color version can be efficiently recovered by randomly re-sampling a new set of Gaussian distributed variables, together with the synthetic grayscale, through the reverse mapping of INNs. To effectively learn the invertible grayscale, we introduce the wavelet transformation into a UNet-like INN architecture, and further present a quantization embedding to prevent the information omission in format conversion, which improves the generalizability of the framework in real-world scenarios. Extensive experiments on three widely used benchmarks demonstrate that the proposed method achieves a state-of-the-art performance in terms of both qualitative and quantitative results, which shows its superiority in multimedia communication and storage systems.
9 February 2023 in IEEE Transactions on Multimedia
Deep learning-based models have achieved remarkable performance in video super-resolution (VSR) in recent years, but most of these models are less applicable to online video applications. These methods solely consider the distortion quality and ignore crucial requirements for online applications, e.g., low latency and low model complexity. In this paper, we focus on online video transmission in which VSR algorithms are required to generate high-resolution video sequences frame by frame in real time. To address such challenges, we propose an extremely low-latency VSR algorithm based on a novel kernel knowledge transfer method, named the convolutional kernel bypass graft (CKBG). First, we design a lightweight network structure that does not require future frames as inputs and saves extra time for caching these frames. Then, our proposed CKBG method enhances this lightweight base model by bypassing the original network with “kernel grafts”, which are extra convolutional kernels containing the prior knowledge of the external pretrained image SR models. During the testing phase, we further accelerate the grafted multibranch network by converting it into a simple single-path structure. The experimental results show that our proposed method can process online video sequences up to 110 FPS with very low model complexity and competitive SR performance.
26 September 2023 in IEEE Transactions on Instrumentation and Measurement
Deep learning-based object detectors in remote sensing (RS) scenarios typically follow the paradigm of pretraining and fine-tuning, to alleviate the limitation of insufficient downstream data. Despite the improved performance, existing pretraining paradigms are sub-optimal due to three deficiencies: 1) inconsistent domains, i.e ., pretraining on natural scenes and fine-tuning for RS scenes, 2) mismatched task objectives, i.e ., classification-oriented pretraining while detection-oriented fine-tuning, and 3) misaligned architectures, i.e ., pretraining only one bare backbone yet neglecting other vital detection components. Against these issues, this paper proposes a novel pretraining paradigm specifically for the task of RS object detection, namely RS strong-classification weak-localization (SCWL) pretraining. Unlike conventional classification pretraining, such as the widely used ImageNet pretraining, our pretraining strategy can adaptively perform bounding box generation on a reconstructed large-scale RS classification-style dataset. These pseudo bounding boxes are integrated with the original accurate class labels as location- and category-related supervisions, respectively, to pretrain the entire RS detectors. The proposed RS SCWL pretraining paradigm is able to significantly improve downstream detection performance and outperforms classification pretraining methods, including ImageNet pretraining. Extensive experiments on different object detection datasets demonstrate its effectiveness and superiority in boosting various RS detectors.
29 May 2024 in IEEE Geoscience and Remote Sensing Letters
Anchor-free object detectors have recently received increasing research attention in the field of aerial scene object detection, due to their high flexibility and practicality. Anchor-free detectors typically depend on the feature pyramid network (FPN) to alleviate the challenge of significant variations in object scales in aerial contexts. Despite establishing a multiscale feature pyramid, existing FPN-based methods treat each aerial object as an indivisible entity solely managed by a single-scale representation. However, they fail to take into account the distinct characteristics of various components within an instance. To this end, this letter proposes a novel anchor-free detector, namely IMPR-Det, which can integrally mix multiscale pyramid representations for different components of an instance, thus boosting the fine-grained object representation capability. Specifically, IMPR-Det fundamentally introduces a more advanced detection head with an adaptive routing mechanism for pixel-level multiscale feature assignment, instead of previous instance-level assignment. Experimental results demonstrate the superiority of the proposed method over its counterparts, in terms of both accuracy and efficiency, for object detection in aerial images.
3 October 2025 in IEEE Journal of Selected Topics in Signal Processing
3D surface representation from multi-view photometric stereo (MVPS) remains a challenging task due to accumulated errors and geometric inconsistencies in conventional fusion-based methods. While 3D Gaussian Splatting (3DGS) enables real-time rendering, its explicit point-based representation struggles to preserve fine geometric details when directly applied to MVPS. To address this limitation, we propose PSGS, a novel photometric-regularized Gaussian splatting framework that integrates surface normal priors to enhance multiview surface projection. Specifically, PS-GS introduces three regularization strategies: (1) normal-guided initialization, which aligns Gaussian splats with photometric surface normals; (2) adaptive splat cloning, which dynamically adjusts point density in regions with high normal variation to improve local detail preservation; and (3) visibility-aware occlusion handling, which ensures geometric consistency across multi-view projections. Unlike NeRF-based MVPS methods, our approach does not rely on implicit volumetric representations and instead optimizes explicit surface projections, achieving efficient and artifact-free rendering. Experimental results on the DiLiGenT-MV benchmark demonstrate that PS-GS significantly improves 2D photometric fidelity, achieving a 2.18 dB PSNR gain over NeRFbased MVPS methods while maintaining around 150× faster rendering efficiency. Additionally, our normal-guided splatting enhances view-consistent normal alignment, reducing artifacts in challenging regions. Although PS-GS does not perform full 3D mesh reconstruction or relighting, its efficient rendering and surface-aware regularization make it a promising approach for real-time multi-view image synthesis in MVPS applications.