Integrating deep learning techniques into the video coding framework gains significant improvement compared to the standard compression techniques, especially applying super-resolution (up-sampling) to down-sampling based video coding as post-processing. However, besides up-sampling degradation, the various artifacts brought from compression make super-resolution problem more difficult to solve. The straightforward solution is to integrate the artifact removal techniques before super-resolution. However, some helpful features may be removed together, degrading the super-resolution performance. To address this problem, we proposed an end-to-end restoration-reconstruction deep neural network (RR-DnCNN) using the degradation-aware technique, which entirely solves degradation from compression and sub-sampling. Besides, we proved that the compression degradation produced by Random Access configuration is rich enough to cover other degradation types, such as Low Delay P and All Intra, for training. Since the straightforward network RR-DnCNN with many layers as a chain has poor learning capability suffering from the gradient vanishing problem, we redesign the network architecture to let reconstruction leverages the captured features from restoration using up-sampling skip connections. Our novel architecture is called restoration-reconstruction u-shaped deep neural network (RR-DnCNN v2.0). As a result, our RR-DnCNN v2.0 outperforms the previous works and can attain 17.02% BD-rate reduction on UHD resolution for all-intra anchored by the standard H.265/HEVC. The source code is available at https://minhmanho.github.io/rrdncnn/.

The increased resolution of Quad Full High Definition (QFHD) offers significantly enhanced visual experience. However, the corresponding huge data throughput of up to 530 Mpixels/s greatly challenges the design of real-time video decoder VLSI with the extensive requirement on both DRAM bandwidth and computational power. In this work, a lossless frame recompression technique and a partial MB reordering scheme are proposed to save the DRAM access of a QFHD video decoder chip. Besides, pipelining and parallelization techniques such as NAL/slice-parallel entropy decoding are implemented to efficiently enhance its computational power. The chip supporting H.264/AVC high profile is fabricated in 90 nm CMOS and verified. It delivers a maximum throughput of 4096×2160@60fps, which is at least 4.3 times higher than the state-of-the-art. DRAM bandwidth requirement is reduced by typically 51%, which fits the design into a 64-bit LPDDR SDRAM interface and results in 58% DRAM power saving. Meanwhile, the core energy is saved by 54% by pipelining and parallelization.

Ultra high definition television (UHDTV) imposes extremely high throughput requirement on video encoders based on High Efficiency Video Coding (H.265/HEVC) and Advanced Video Coding (H.264/AVC) standards. Context-adaptive binary arithmetic coding (CABAC) is the entropy coding component of these standards. In very-large-scale integration implementation, CABAC has known difficulties in being effectively pipelined and parallelized, due to the critical bin-to-bin data dependencies in its algorithm. This paper addresses the throughput requirement of CABAC encoding for UHDTV applications. The proposed optimizations including prenormalization, hybrid path coverage and lookahead rLPS to reduce the critical path delay of binary arithmetic encoding (BAE) by exploiting the incompleteness of data dependencies in rLPS updating. Meanwhile, the number of bins BAE delivers per clock cycle is increased by the proposed bypass bin splitting technique. The context modeling and binarization components are also optimized. As a result, our CABAC encoder delivers an average of 4.37 bins per clock cycle. Its maximum clock frequency reaches 420 MHz when synthesized in 90 nm. The corresponding overall throughput is 1836 Mbin/s that is 62.5% higher than the state-of-the-art architecture.

3840 × 2160 and 7680 × 4320 UHDTV formats deliver remarkably enhanced visual experience relative to high definition but in the meanwhile involve huge complexity and memory bandwidth requirements in video encoding. Especially, enlarged motion distances of UHDTV lead to additional difficulties in the implementation of motion estimation, which is originally the most critical bottleneck of an encoder. This paper presents a motion estimation processor design for H.264/AVC. A test chip is implemented in 40 nm CMOS. With algorithm and architecture co-optimization, the processor delivers a maximum throughput of 1.59 Gpixel/s for 7680 × 4320 48 fps video, at least 7.5 times faster than previous designs. The corresponding core power dissipation is 622 mW at 210 MHz, with energy efficiency improved by at least 23%. The chip's DRAM bandwidth requirement is also 68% lower than previous chips. With a maximum search range of ±211 (horizontal) by ±106 (vertical) around a predictive search center, the proposed motion estimation processor well accommodates the high motion of UHDTV.

8K Ultra HD is being promoted as the next-generation digital video format. From a communication channel perspective, the latest high-efficiency video coding standard (H.265/HEVC) greatly enhances the feasibility of 8K by doubling the compression ratio. Implementation of such codecs is a challenge, owing to ultra-high throughput requirements and increased complexity per pixel. The former corresponds to up to 10b/pixel, 7680×4320pixels/frame and 120fps - 80× larger than 1080p HD. The latter comes from the new features of HEVC relative to its predecessor H.264/AVC. The most challenging of them is the enlarged and highly variable-size coding/prediction/transform units (CU/PU/TU), which significantly increase: 1) the requirement for on-chip memory as pipeline buffers, 2) the difficulty in maintianing pipeline utilization, and 3) the complexity of inverse transforms (IT). This paper presents an HEVC decoder chip supporting 8K Ultra HD, featuring a 16pixel/cycle true-variable-block-size system pipeline. The pipeline: 1) saves on-chip memory with a novel block-in-block-out (BIBO) queue system and a parameter delivery network, and 2) allows high design efficiency and utilization of processing components through local synchronization. Key optimizations at the component level are also presented.

8K ultra-HD is being promoted as the next-generation video specification. While the High Efficiency Video Coding (HEVC) standard greatly enhances the feasibility of 8K with a doubled compression ratio, its implementation is a challenge, owing to ultrahigh-throughput requirements and increased complexity per pixel. The latter comes from the new features of HEVC. At the system level, the most challenging of them is the enlarged and highly variable-size coding/prediction/transform units, which significantly increase the requirement for on-chip memory as pipeline buffers and the difficulty in maintaining pipeline utilization. This paper presents an HEVC decoder chip featuring a system pipeline that works at a nonunified and variable granularity. The pipeline saves on-chip memory with a novel block-in-block-out queue system and a parameter delivery network, while allowing overhead-free and fully pipelined operation of the processing components. With the system pipeline design combined with various component-level optimizations, the proposed decoder in 40 nm achieves a maximum throughput of 4 Gpixels/s or 8K 120 frames/s for the low-delay-P configuration of HEVC, 7.5–55 times faster than prior works. It supports 8K 60 frames/s for the low-delay and random-access configurations. In a normalized comparison, it also shows 3.1–3.6 times better area efficiency and 31%–55% superior energy efficiency.