9 November 2023 in IEEE Transactions on Multimedia
This paper develops a Scalable Point Cloud Attribute Compression solution, termed ScalablePCAC . In a two-layer example, ScalablePCAC uses the standard G-PCC at the base layer to directly encode the thumbnail point cloud that is downscaled from the original input, and a learning-based model at the enhancement layer to compress and restore the full-resolution input point cloud conditioned on the base layer reconstruction. As such, the base layer provides a coarse reconstruction of the input point cloud and the enhancement layer further improves the quality. We then adopt a cross-layer rate allocation strategy that flexibly determines the resolution downscaling factor, the quantization parameter of the base layer, and the quality controlling factor of the enhancement layer to adapt the bitrate of the two layers for approximately optimal Rate-Distortion (R-D) performance. We conduct extensive experiments on popular point clouds following the MPEG common test conditions. Results demonstrate that the proposed ScalablePCAC achieves $>$ 10% BD-BR reduction against the latest G-PCC version 22 (TMC13v22) on the Y component; it also significantly outperforms existing learning-based solutions for point cloud attribute compression, e.g. , compared with a recent work showing state-of-the-art performance, it achieves $>$ 20% BD-BR reduction.
11 October 2024 in IEEE Transactions on Emerging Topics in Computational Intelligence
We propose DeepPCC, an end-to-end learning-based approach for the lossy compression of large-scale object point clouds. For both geometry and attribute components, we introduce the Multiscale Neighborhood Information Aggregation (NIA) mechanism, which applies resolution downscaling progressively ( i.e. , dyadic downsampling of geometry and average pooling of attribute) and combines sparse convolution and local self-attention at each resolution scale for effective feature representation. Under a simple autoencoder structure, scale-wise NIA blocks are stacked as the analysis and synthesis transform in the encoder-decoder pair to best characterize spatial neighbors for accurate approximation of geometry occupancy probability and attribute intensity. Experiments demonstrate that DeepPCC remarkably outperforms state-of-the-art rules-based MPEG G-PCC and learning-based solutions both quantitatively and qualitatively, providing strong evidence that DeepPCC is a promising solution for emerging AI-based PCC.
13 May 2024 in IEEE Transactions on Circuits and Systems for Video Technology
The Geometry-based Point Cloud Compression (G-PCC) standard enables point cloud delivery over the internet through efficient compression. Limited by the transmission bandwidth, rate control is demanded in G-PCC for high-quality point cloud video streaming. This paper thus proposes a content-aware rate control solution for G-PCC. Given the target bitrate and distortion evaluation criteria, our method can predict the geometry and attribute quantizers for G-PCC while minimizing the overall distortion. Specifically, as the rate and distortion of both geometry and attribute are involved in G-PCC, we separately establish rate/distortion models for geometry and attribute. Moreover, recognizing the dependence of attribute compression on reconstructed geometry, we integrate the geometry quantizer into the attribute rate/distortion models to improve prediction accuracy. For dynamic coding scenarios, we leverage selective representative frames for efficient model parameter initialization. Additionally, we introduce a μ updating strategy that dynamically incorporates information from previous frames to update the existing models. Extensive experiments demonstrate the effectiveness of our proposed method. Under the G-PCC common test condition, our method achieves remarkable rate accuracy, with a 5.3% bitrate error for static coding and 0.3% for dynamic coding. Moreover, it achieves >15% BD-Rate gains over the G-PCC anchor. These results showcase its capabilities in delivering high-fidelity point cloud video streams within the bandwidth constraint.
12 March 2024 in IEEE Transactions on Visualization and Computer Graphics
The emergence of holographic media drives the standardization of Geometry-based Point Cloud Compression (G-PCC) to sustain networked service provisioning. However, G-PCC inevitably introduces visually annoying artifacts, degrading the quality of experience (QoE). This work focuses on restoring G-PCC compressed point cloud attributes, e.g., RGB colors, to which fully data-driven and rules-unrolling-based post-processing filters are studied. At first, as compressed attributes exhibit nested blockiness, we develop a learning-based sample adaptive offset (NeuralSAO), which leverages a neural model using multiscale feature aggregation and embedding to characterize local correlations for quantization error compensation. Later, given statistically Gaussian distributed quantization noise, we suggest the utilization of a bilateral filter with Gaussian kernels to weigh neighbors by jointly considering their geometric and photometric contributions for restoration. Since local signals often present varying distributions, we propose estimating the smoothing parameters of the bilateral filter using an ultra-lightweight neural model. Such a bilateral filter with learnable parameters is called NeuralBF. The proposed NeuralSAO demonstrates the state-of-art restoration quality improvement, e.g. , >20% BD-BR (Bjøntegaard delta rate) reduction over G-PCC on solid points clouds. However, NeuralSAO is computationally intensive and may suffer from poor generalization. On the other hand, although NeuralBF only achieves half of the gains of NeuralSAO, it is lightweight and exhibits impressive generalization across various samples. This comparative study between the data-driven large-scale NeuralSAO and the rules-unrolling-based small-scale NeuralBF helps to understand the capacity ( i.e. , performance, complexity, generalization) of underlying filters in terms of the quality restoration for compressed point cloud attribute.
8 December 2025 in IEEE Transactions on Image Processing
Point cloud video streaming is promising for immersive media applications, which urges the development of efficient compression methods. However, existing approaches either suffer from poor performance or lack effective coder control mechanisms, making them impractical for networked point cloud services, where bandwidth is often constrained and fluctuates over time. Therefore, this paper proposes a system-level solution – a layered point cloud compressor, called Yak, to address these issues. Yak offers comprehensive support for both intra and inter-frame coding of geometry and attribute components in point cloud sequences. It consists of three layers: the Base Layer uses the standard G-PCC to encode a thumbnail counterpart downscaled from the input point cloud; the Enhancement Layer devises the end-to-end variational autoencoder to compress the original input conditioned on the base layer reconstruction, and the Dynamic Layer generates feature-space predictions as the temporal prior for conditional inter-frame coding. In addition, Yak devises the Content Analysis module to dynamically determine the optimal encoding parameters of each frame, by which bits budget is intelligently allocated for geometry and attribute components to maximize the overall rate-distortion (R-D) performance. Such accurate rate control relies on the parametric rate/distortion models whose parameters are initialized through one-pass template matching and frame-wise delta updating constrained by R-D optimization. Following standard evaluation guidelines, Yak has notably outperformed traditional rules-based methods such as MPEG G-PCC and V-PCC, as well as other learning-based approaches, while offering flexible networked adaption and affordable complexity.