Li Chen

Li Chen

Auto-generated profile, based on publicly available information

Li Chen

Shanghai Jiao Tong University China

Research Focus & Metrics

Academic Career Length

18Years (2009-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 April 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
GLGuo Lu
Guo Lu
XZXiaoyun Zhang
Xiaoyun Zhang
WOWanli Ouyang
Wanli Ouyang
LCLi Chen
Li Chen
ZGZhiyong Gao
Zhiyong Gao
DXDong Xu
Dong Xu
Traditional video compression approaches build upon the hybrid coding framework with motion-compensated prediction and residual transform coding. In this paper, taking advantage of both the classical compression architecture and the powerful non-linear representation ability of neural networks, we propose the first end-to-end deep video compression framework. Our framework employs pixel-wise motion information, which is learned from an optical flow network and further compressed by an auto-encoder network to save bits. The other compression components are also implemented by well-designed networks for high efficiency. All the modules are jointly optimized by using the rate-distortion trade-off and collaborate with each other. More importantly, the proposed deep video compression framework is very flexible and can be easily extended by using lightweight or advanced networks for higher speed or better efficiency. Experimental results show that the proposed approach can outperform the widely used video coding standard H.264 and be even on par with the latest standard H.265.

3 November 2020
Verified
Book Chapter
Closed access
PDF file
GLGuo Lu
Guo Lu
CCChunlei Cai
Chunlei Cai
XZXiaoyun Zhang
Xiaoyun Zhang
LCLi Chen
Li Chen
WOWanli Ouyang
Wanli Ouyang
ZGZhiyong Gao
Zhiyong Gao

25 December 2019 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
CCChunlei Cai
Chunlei Cai
LCLi Chen
Li Chen
XZXiaoyun Zhang
Xiaoyun Zhang
ZGZhiyong Gao
Zhiyong Gao
Compressing an image with more bits automatically allocated to the region of interest (ROI) than to the background can both protect key information and reduce substantial redundancy. This paper models ROI image compression as an optimization problem of minimizing a weighted sum of the rate of the image and distortion of the ROI. The traditional framework solves this problem by cascading ROI prediction and ROI coding, through which achieving the optimized solution is impossible. To improve coding performance, we propose a novel deep-learning-based unified framework that can achieve rate distortion optimization for ROI compression. Specifically, the proposed framework includes a pair of ROI encoder and decoder convolutional neural networks and a learned entropy codec. The encoder network simultaneously generates multiscale representations that support efficient rate allocation and an implicit ROI mask that guides rate allocation. The proposed framework can automatically complete ROI image compression, and it can be optimized from data in an end-to-end manner. To effectively train the framework by back propagation, we develop a soft-to-hard ROI prediction scheme to make the entire framework differential. To improve visual quality, we propose a hierarchical distortion loss function to protect both pixel-level fidelity for ROI and structural similarity for the entire image. The proposed framework is implemented in two scenarios: salient-target and face-target ROI compression. Comparative experiments demonstrate the advantages of the proposed framework over the traditional framework, including considerably better subjective visual quality, significantly higher objective ROI compression performance and execution efficiency.

12 November 2018 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
CCChunlei Cai
Chunlei Cai
LCLi Chen
Li Chen
XZXiaoyun Zhang
Xiaoyun Zhang
ZGZhiyong Gao
Zhiyong Gao
While deep learning image compression methods have shown impressive coding performance, most of them output a single optimized compression rate using a trained specific network. However, in practical it is essential to support variable rate compression or meet a target rate with high coding performance. This paper proposes a novel image compression method, making it possible for a single CNN model to generate variable rate efficiently with optimized rate-distortion (RD) performance. The method consists of CNN based multi-scale decomposition transform and content adaptive rate allocation. Specifically, the transform network is learned to decompose the input image into several scales of representations while optimizing the RD performance for all scales. Rate allocation algorithms for two typical scenarios are provided to determine the optimal scale of each image block for a given target rate or quality-factor. For a target rate, the allocation is adaptive based on content complexity. And for a target quality-factor which indicates a trade-off between rate and quality, the optimal scale is determined by minimizing the RD cost. Experimental results have shown that our method has outperformed JPEG2000 and BPG standards with high efficiency and state-of-the-art RD performance as measured by MS-SSIM. Moreover, our method can strictly control the rate to generate the target compression result.

16 November 2020 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
WSWang Shen
Wang Shen
WBWenbo Bao
Wenbo Bao
GZGuangtao Zhai
Guangtao Zhai
LCLi Chen
Li Chen
XMXiongkuo Min
Xiongkuo Min
ZGZhiyong Gao
Zhiyong Gao
Video frame interpolation aims to improve users’ watching experiences by generating high-frame-rate videos from low-frame-rate ones. Existing approaches typically focus on synthesizing intermediate frames using high-quality reference images. However, the captured reference frames may suffer from inevitable spatial degradations such as motion blur, sensor noise, etc. Few studies have approached the joint video enhancement problem, namely synthesizing high-frame-rate and high-quality results from low-frame-rate degraded inputs. In this paper, we propose a unified optimization framework for video frame interpolation with spatial degradations. Specifically, we develop a frame interpolation module with a pyramid structure to cyclically synthesize high-quality intermediate frames. The pyramid module features adjustable spatial receptive field and temporal scope, thus contributing to controllable computational complexity and restoration ability. Besides, we propose an inter-pyramid recurrent module to connect sequential models to exploit the temporal relationship. The pyramid module integrates the recurrent module, thus can iteratively synthesize temporally smooth results. And the pyramid modules share weights across iterations, thus it does not expand the model’s parameter size. Our model can be generalized to several applications such as up-converting the frame rate of videos with motion blur, reducing compression artifacts, and jointly super-resolving low-resolution videos. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art methods on various video frame interpolation and enhancement tasks.

20 February 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YTYuan Tian
Yuan Tian
GLGuo Lu
Guo Lu
Yichao YanYichao Yan
Yichao Yan
GZGuangtao Zhai
Guangtao Zhai
LCLi Chen
Li Chen
ZGZhiyong Gao
Zhiyong Gao
Video compression is indispensable to most video analysis systems. Despite saving the transportation bandwidth, it also deteriorates downstream video understanding tasks, especially at low-bitrate settings. To systematically investigate this problem, we first thoroughly review the previous methods, revealing that three principles, i.e. , task-decoupled, label-free, and data-emerged semantic prior, are critical to a machine-friendly coding framework but are not fully satisfied so far. In this paper, we propose a traditional-neural mixed coding framework that simultaneously fulfills all these principles, by taking advantage of both traditional codecs and neural networks (NNs). On one hand, the traditional codecs can efficiently encode the pixel signal of videos but may distort the semantic information. On the other hand, highly non-linear NNs are proficient in condensing video semantics into a compact representation. The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved w.r.t. the coding procedure, which is spontaneously learned from unlabeled data in a self-supervised manner. The videos collaboratively decoded from two streams (codec and NN) are of rich semantics, as well as visually photo-realistic, empirically boosting several mainstream downstream video analysis task performances without any post-adaptation procedure. Furthermore, by introducing the attention mechanism and adaptive modeling scheme, the video semantic modeling ability of our approach is further enhanced. Fianlly, we build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach. All codes, data, and models will be open-sourced for facilitating future research.