Ruihao Gong

Dr. Ruihao Gong

orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 September 2020 in Pattern Recognition
Verified
Journal Article
Closed access
HQHaotong Qin
Haotong Qin
RGRuihao Gong
Ruihao Gong
Xianglong LiuXianglong Liu
Xianglong Liu
XBXiao Bai
Xiao Bai
JSJingkuan Song
Jingkuan Song
NSNicu Sebe
Nicu Sebe

1 October 2019
Verified
Conference Paper
Closed access
RGRuihao Gong
Ruihao Gong
XLXianglong Liu
Xianglong Liu
SJShenghu Jiang
Shenghu Jiang
TLTianxiang Li
Tianxiang Li
PHPeng Hu
Peng Hu
JYJunjie Yan
Junjie Yan

1 June 2020
Verified
Conference Paper
Closed access
HQHaotong Qin
Haotong Qin
RGRuihao Gong
Ruihao Gong
XLXianglong Liu
Xianglong Liu
MSMingzhu Shen
Mingzhu Shen
ZWZiran Wei
Ziran Wei
JSJingkuan Song
Jingkuan Song
Weight and activation binarization is an effective approach to deep neural network compression and can accelerate the inference by leveraging bitwise operations. Although many binarization methods have improved the accuracy of the model by minimizing the quantization error in forward propagation, there remains a noticeable performance gap between the binarized model and the full-precision one. Our empirical study indicates that the quantization brings information loss in both forward and backward propagation, which is the bottleneck of training accurate binary neural networks. To address these issues, we propose an Information Retention Network (IR-Net) to retain the information that consists in the forward activations and backward gradients. IR-Net mainly relies on two technical contributions: (1) Libra Parameter Binarization (Libra-PB): simultaneously minimizing both quantization error and information loss of parameters by balanced and standardized weights in forward propagation; (2) Error Decay Estimator (EDE): minimizing the information loss of gradients by gradually approximating the sign function in backward propagation, jointly considering the updating ability and accurate gradients. We are the first to investigate both forward and backward processes of binary networks from the unified information perspective, which provides new insight into the mechanism of network binarization. Comprehensive experiments with various network structures on CIFAR-10 and ImageNet datasets manifest that the proposed IR-Net can consistently outperform state-of-the-art quantization methods.

1 January 2023
Verified
Conference Paper
Open access
PDF file
XWXiuying Wei
Xiuying Wei
YZYunchen Zhang
Yunchen Zhang
YLYuhang Li
Yuhang Li
XZXiangguo Zhang
Xiangguo Zhang
RGRuihao Gong
Ruihao Gong
Xianglong LiuXianglong Liu
Xianglong Liu
Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, Xianglong Liu. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

1 June 2020
Verified
Conference Paper
Closed access
FZFeng Zhu
Feng Zhu
RGRuihao Gong
Ruihao Gong
FYFengwei Yu
Fengwei Yu
XLXianglong Liu
Xianglong Liu
YWYanfei Wang
Yanfei Wang
JYJunjie Yan
Junjie Yan
Recently low-bit (e.g., 8-bit) network quantization has been extensively studied to accelerate the inference. Besides inference, low-bit training with quantized gradients can further bring more considerable acceleration, since the backward process is often computation-intensive. Unfortunately, the inappropriate quantization of backward propagation usually makes the training unstable and even crash. There lacks a successful unified low-bit training framework that can support diverse networks on various tasks. In this paper, we give an attempt to build a unified 8-bit (INT8) training framework for common convolutional neural networks from the aspects of both accuracy and speed. First, we empirically find the four distinctive characteristics of gradients, which provide us insightful clues for gradient quantization. Then, we theoretically give an in-depth analysis of the convergence bound and derive two principles for stable INT8 training. Finally, we propose two universal techniques, including Direction Sensitive Gradient Clipping that reduces the direction deviation of gradients and Deviation Counteractive Learning Rate Scaling that avoids illegal gradient update along the wrong direction. The experiments show that our unified solution promises accurate and efficient INT8 training for a variety of networks and tasks, including MobileNetV2, InceptionV3 and object detection that prior studies have never succeeded. Moreover, it enjoys a strong flexibility to run on off-the-shelf hardware, and reduces the training time by 22% on Pascal GPU without too much optimization effort. We believe that this pioneering study will help lead the community towards a fully unified INT8 training for convolutional neural networks.

2 October 2022 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
Haotong QinHaotong Qin
Haotong Qin
XZXiangguo Zhang
Xiangguo Zhang
RGRuihao Gong
Ruihao Gong
YDYifu Ding
Yifu Ding
YXYi Xu
Yi Xu
Xianglong LiuXianglong Liu
Xianglong Liu