25 January 2022 in IEEE Transactions on Consumer Electronics
Versatile Video Coding (VVC) was finalized in 2020 and offered promising coding efficiency with a bitrate reduction of about 50% for same video quality as High Efficiency Video Coding. However, its high encoding complexity is a heavy burden on real-time applications. In particular, the very high complexity in intra coding can be a big barrier into market entry. This paper presents an efficient low-complexity intra coding scheme, which employs downsampling and upsampling processes. The downsampling is simply performed by reducing the resolution of an original video in both horizontal and vertical directions. In the upsampling, convolutional neural network based super-resolution is used to increase the resolution of the reconstructed video. In addition, this paper thoroughly analyzes the performance and complexity of all intra coding tools in VVC. Experimental results demonstrate that the significantly high reduction of the encoding complexity can be achieved with acceptable video quality.
8 April 2025 in Engineering Applications of Artificial Intelligence
28 March 2023 in IEEE MultiMedia
Immersive multimedia has received a lot of attention because of its huge impact on user experience. In order to realize high immersion in virtual environments, many virtual views should be generated at arbitrary viewpoints with advanced display devices. However, specular regions, which affect user experience, have not been fully investigated in an immersive multimedia field. In this article, we propose specular highlight detection and rendering methods to improve the immersion. For the specular detection, a high-performance variational attention U-network (VAUnet), which combines a variational autoencoder and a spatial attention mechanism, is proposed with a hybrid loss function. The specular regions detected from VAUnet are compressed with an immersive video coding standard (MPEG-I), and then the rendering is performed by considering the decompressed specular regions. Extensive experiments demonstrate that the proposed method improves specular detection accuracy and subjective rendering quality.
16 December 2024 in IEEE MultiMedia
Image denoising is a process of removing noise from an image to improve its quality and clarity. Many state-of-the-art networks utilize real-valued representation, which can miss cross-correlation of RGB channels in a color image. To tackle this limitation, this paper proposes Quaternion Transformer-based image denoising (QformerID). Specifically, it includes a novel Quaternion Transformer layer (QTL) that effectively learns long-range correlations between pixels and preserves the cross-correlation of RGB channels, based on Quaternion representation. Experimental results show that QformerID offers higher performance on various datasets including Gaussian and real-world noisy images in terms of objective and subjective qualities, compared with state-of-the-art networks. In addition, visual assessment shows that QformerID not only preserves image details but also suppresses noise very well.
1 January 2026 in IEEE Access
18 March 2024 in IEEE Access
Video captioning is an automatic task that collects natural language to represent visual content. Recently, it has achieved lots of amazing progress thanks to deep learning techniques. Most techniques have mainly focused on a deep learning network architecture, whereas video quality and resolution have not been fully considered, although their impact on captioning performance is very strong. Since video communication systems usually perform compression, original quality and resolution can be degraded and down-sampled for significant reduction of the data size, which results in severe quality degradation. Hence, this paper analyzes the impact of the compression and the down-sampling on the captioning, and proposes a quality enhancement method using for the video captioning. First, the proposed method performs quality classification to analyze the quality of each frame. Next, super-resolution (SR) is used to enhance the frames in terms of their quality and resolution. Finally, a video captioning network uses the enhanced frames to generate accurate sentences. Experimental results show that the proposed method drastically improves the captioning performance, when both quality and resolution of input videos are randomly determined.