Li Zhang

Li Zhang

Auto-generated profile, based on publicly available information

Li Zhang

ByteDance Inc., Beijing, China

Research Focus & Metrics

Academic Career Length

18Years (2009-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 July 2021 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Open access
WCWei-Jung Chien
Wei-Jung Chien
LZLi Zhang
Li Zhang
MWMartin Winken
Martin Winken
XLXiang Li
Xiang Li
RLRu-Ling Liao
Ru-Ling Liao
CCChun-Chi Chen
Chun-Chi Chen
This paper overviews the motion vector coding and block merging techniques in the Versatile Video Coding (VVC) standard developed by the Joint Video Experts Team (JVET). In general, inter-prediction techniques in VVC can be classified into two major groups: “whole block-based inter prediction” and “subblock-based inter prediction”. In this paper, we focus on techniques for whole block-based inter prediction. As in its predecessor, High Efficiency Video Coding (HEVC), whole block-based inter prediction in VVC is represented by adaptive motion vector prediction (AMVP) mode or merge mode. Newly introduced features purely for AMVP mode include symmetric motion vector difference and adaptive motion vector resolution. The features purely for merge mode include pairwise average merge, merge with motion vector difference, combined inter-intra prediction and geometric partitioning mode. Coding tools such as history-based motion vector prediction and bidirectional prediction with coding unit weights can be applied on both AMVP mode and merge mode. This paper discusses the design principles and the implementation of the new inter-prediction methods. Using objective metrics, simulation results show that the methods overviewed in the paper can jointly achieve 6.2% and 4.7% BD-rate savings on average with the random access and low-delay configurations, respectively. Significant subjective picture quality improvements of some tools are also reported when comparing the resulting pictures at same bitrates.

28 November 2019 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
MWMeng Wang
Meng Wang
JLJunru Li
Junru Li
LZLi Zhang
Li Zhang
KZKai Zhang
Kai Zhang
HLHongbin Liu
Hongbin Liu
SMSiwei Ma
Siwei Ma
The flexible partitioning structure such as quad-tree plus binary-tree (QTBT) plays important roles in the next generation video coding standard. This paper investigates new coding unit (CU) partitioning methods to further improve the compression efficiency. In particular, we propose the extended quad-tree partitioning and asymmetric ternary-tree partitioning to further develop the principle behind the existing partitioning structure. First, a central extended quad-tree (CENTRAL-EQT) partitioning is introduced, which extends the traditional QT partitioning with central pattern and generates four elaborately designed sub-CUs with different sizes. Second, we propose a parallel extended quad-tree (PARALLEL-EQT) partitioning that allows the CU to split along a single direction, leading to four identical size sub-blocks. Third, we present an asymmetric ternary-tree (ATT) method, which splits the CU asymmetrically into three sub-blocks. The proposed new partitioning methods allow the interleaving with binary-tree partitioning for enhanced adaptability, and they can be jointly enabled to capture different characteristics of the local content. Simulation results on the JEM7-QTBT-Only platform show that averagely 4.92%, 4.81% and 5.08% BD-Rate savings are achieved for the luma component by the proposed methods under random access (RA), low-delay P (LDP) and low-delay B (LDB) configurations, respectively, with around 725% encoding time increase and negligible decoding time increase. Furthermore, experimental results also reveal that the proposed partitioning schemes are effective when cooperating with the upcoming Versatile Video Coding standard.

11 September 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
ZZZhaobin Zhang
Zhaobin Zhang
SESemih Esenlik
Semih Esenlik
YWYaojun Wu
Yaojun Wu
MWMeng Wang
Meng Wang
KZKai Zhang
Kai Zhang
LZLi Zhang
Li Zhang
The autoregressive model has been widely used in learning-based image compression due to its superior context modeling capability. However, its sequential processing nature also undermines the ability of decoding in parallel and hinders the deployment in real applications. In this paper, we propose a decoupled framework to resolve this issue. With the decoupled architecture, the entropy decoding process is independent of the latent sample reconstruction process. The entropy decoding process thus can be finished before the latent sample prediction process begins, which leads to significant decoding time savings by enabling the two processes to be conducted in parallel. To further reduce the decoding time, we introduce wavefront processing, where multiple rows can be processed simultaneously when reconstructing the latent samples. On top of that, we design a series of coding tools to improve the rate-distortion efficiency and reduce the decoding complexity. Device interoperability is also supported by the proposed solution, where the same bitstream can be successfully decoded on different CPU/GPU devices. Comprehensive experiments are conducted to validate the effectiveness of the proposed method. Using objective evaluation metrics required by JPEG AI Call for Proposals (CfP), the proposed method achieves an average of -29.6% BD-rate changes with 2.44 times faster decoding speed compared to VVC image coding. When compared to the commonly used benchmark learning-based methods, the proposed method achieves -30.5% BD-rate changes and 101 times faster decoding speed over cheng2020attn. The proposed solution has been proposed to JPEG AI and IEEE 1857.11 as a response to CfP and the core techniques have been adopted to build the verification model. The software and the instructions can be accessed at https://github.com/bytedance/BEE.

26 August 2016 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Verified
Journal Article
Closed access
LZLi Zhang
Li Zhang
XXXiaoyu Xiu
Xiaoyu Xiu
JCJianle Chen
Jianle Chen
MKMarta Karczewicz
Marta Karczewicz
YHYunwen He
Yunwen He
WKWoo-Shik Kim
Woo-Shik Kim
The screen content coding (SCC) Extensions of high efficiency video coding (HEVC) employs in-loop adaptive color-space transform (ACT) technique to explore the inter-color-component redundancy, i.e., statistical redundancy among different color components. In ACT, the prediction residual signal is adaptively converted into a different color space, i.e., YCgCo. The rate-distortion criteria is employed to decide whether to code the residual signal in the original color space or YCgCo color space. Typically, the inter-color-component correlation could be reduced when ACT is enabled. The residual signal after possible color-space conversion is then coded, following the existing HEVC framework, i.e., transform if necessary, quantization and entropy coded. This paper describes the design of ACT from several points of view, from theoretical analysis to implementation details. Experimental results are also provided to demonstrate the significant coding gains of ACT in the HEVC SCC Extensions.

14 April 2020 in IEEE Journal of Selected Topics in Signal Processing
Verified
Journal Article
Closed access
JLJiguo Li
Jiguo Li
XZXinfeng Zhang
Xinfeng Zhang
CJChuanmin Jia
Chuanmin Jia
JXJizheng Xu
Jizheng Xu
LZLi Zhang
Li Zhang
WGWen Gao
Wen Gao
Direct speech-to-image translation without text is an interesting and useful topic due to the potential applications in human-computer interaction, art creation, computer-aided design. etc. Not to mention that many languages have no writing form. However, as far as we know, it has not been well-studied how to translate the speech signals into images directly and how well they can be translated. In this paper, we attempt to translate the speech signals into the image signals without the transcription stage. Specifically, a speech encoder is designed to represent the input speech signals as an embedding feature, and it is trained with a pretrained image encoder using teacher-student learning to obtain better generalization ability on new classes. Subsequently, a stacked generative adversarial network is used to synthesize high-quality images conditioned on the embedding feature. Experimental results on both synthesized and real data show that our proposed method is effective to translate the raw speech signals into images without the middle text representation. Ablation study gives more insights about our method.

23 January 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
YLYue Li
Yue Li
LZLi Zhang
Li Zhang
KZKai Zhang
Kai Zhang
As a rapid development of neural-network-based machine learning algorithms, deep learning methods are being tentatively used in a much wider range than well-known artificial intelligence applications such as face recognition or auto-driving. Recently, deep learning models are investigated intensively to improve the compression efficiency for video coding, especially at the in-loop filtering stage. Although deep learning-based in-loop filtering methods in prior arts have already shown a remarkable potential capability in video coding, content propagation issue is still not well recognized and addressed yet. Content propagation is the fact that contents of reference frames are propagated to frames referring to them, which typically leads to over-filtering issues. In this paper, we develop an iteratively trained deep in-loop filter with adaptive model selection (iDAM) to address the content propagation issue. First, we propose an iterative training scheme, which enables the network to gradually take into account the impacts of content propagation. Second, we propose a filter selection mechanism, i.e. allowing a block to select from a set of candidate filters with different filtering strengths. Besides, we propose a novel approach to design a conditional in-loop filtering method that can deal with multiple quality levels with a single model and serve the functionality of filter selection by modifying the input parameters. Extensive experiments on top of the latest video coding standard (Versatile Video Coding, VVC) have been conducted to evaluate the proposed techniques. Compared with VTM-11.0, our scheme achieves a new state-of-the-art, leading to {7.91%, 20.25%, 20.44%}, {11.64%, 26.40%, 26.50%}, and {10.97%, 26.63%, 26.77%} BD-rate reductions on average for {Y, Cb, Cr} under all-intra, random-access, and low-delay configurations respectively. As far as we know, our proposed iDAM scheme provides the highest coding performance compared to all existing solutions. In addition, the syntax elements of the proposed scheme were adopted at the 76th meeting of Audio Video coding Standard (AVS) held this year.