Shuai Zhao

Shuai Zhao

Auto-generated profile, based on publicly available information

Shuai Zhao

ReLER Lab Zhejiang University CCAI

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 December 2024 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
SZShuai Zhao
Shuai Zhao
RQRuijie Quan
Ruijie Quan
LZLinchao Zhu
Linchao Zhu
YYYi Yang
Yi Yang
Pre-trained vision-language models (VLMs) are the de-facto foundation models for various downstream tasks. However, scene text recognition methods still prefer backbones pre-trained on a single modality, namely, the visual modality, despite the potential of VLMs to serve as powerful scene text readers. For example, CLIP can robustly identify regular (horizontal) and irregular (rotated, curved, blurred, or occluded) text in images. With such merits, we transform CLIP into a scene text reader and introduce CLIP4STR, a simple yet effective STR method built upon image and text encoders of CLIP. It has two encoder-decoder branches: a visual branch and a cross-modal branch. The visual branch provides an initial prediction based on the visual feature, and the cross-modal branch refines this prediction by addressing the discrepancy between the visual feature and text semantics. To fully leverage the capabilities of both branches, we design a dual predict-and-refine decoding scheme for inference. We scale CLIP4STR in terms of the model size, pre-training data, and training data, achieving state-of-the-art performance on 13 STR benchmarks. Additionally, a comprehensive empirical study is provided to enhance the understanding of the adaptation of CLIP to STR. We believe our method establishes a simple yet strong baseline for future STR research with VLMs.

5 September 2022 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
SZShuai Zhao
Shuai Zhao
LZLiguang Zhou
Liguang Zhou
WWWenxiao Wang
Wenxiao Wang
DCDeng Cai
Deng Cai
TLTin Lun Lam
Tin Lun Lam
YXYangsheng Xu
Yangsheng Xu
The width of a neural network matters since increasing the width will necessarily increase the model capacity. However, the performance of a network does not improve linearly with the width and soon gets saturated. In this case, we argue that increasing the number of networks (ensemble) can achieve better accuracy-efficiency trade-offs than purely increasing the width. To prove it, one large network is divided into several small ones regarding its parameters and regularization components. Each of these small networks has a fraction of the original one’s parameters. We then train these small networks together and make them see various views of the same data to increase their diversity. During this co-training process, networks can also learn from each other. As a result, small networks can achieve better ensemble performance than the large one with few or no extra parameters or FLOPs, i . e ., achieving better accuracy-efficiency trade-offs. Small networks can also achieve faster inference speed than the large one by concurrent running. All of the above shows that the number of networks is a new dimension of model scaling. We validate our argument with 8 different neural architectures on common benchmarks through extensive experiments.

31 December 2020 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
DSDong Shen
Dong Shen
SZShuai Zhao
Shuai Zhao
JHJinming Hu
Jinming Hu
HFHao Feng
Hao Feng
DCDeng Cai
Deng Cai
XHXiaofei He
Xiaofei He
As an instance-level recognition problem, re-identification (re-ID) requires models to capture diverse features. However, with continuous training, re-ID models pay more and more attention to the salient areas. As a result, the model may only focus on few small regions with salient representations and ignore other important information. This phenomenon leads to inferior performance, especially when models are evaluated on small inter-identity variation data. In this paper, we propose a novel network, Erasing-Salient Net (ES-Net), to learn comprehensive features by erasing the salient areas in an image. ES-Net proposes a novel method to locate the salient areas by the confidence of objects and erases them efficiently in a training batch. Meanwhile, to mitigate the over-erasing problem, this paper uses a trainable pooling layer P-pooling that generalizes global max and global average pooling. Experiments are conducted on two specific re-identification tasks (i.e., Person re-ID, Vehicle re-ID). Our ES-Net outperforms state-of-the-art methods on three Person re-ID benchmarks and two Vehicle re-ID benchmarks. Specifically, mAP / Rank-1 rate: 88.6% / 95.7% on Market1501, 78.8% / 89.2% on DuckMTMC-reID, 57.3% / 80.9% on MSMT17, 81.9% / 97.0% on Veri-776, respectively. Rank-1 / Rank-5 rate: 83.6% / 96.9% on VehicleID (Small), 79.9% / 93.5% on VehicleID (Medium), 76.9% / 90.7% on VehicleID (Large), respectively. Moreover, the visualized salient areas show human-interpretable visual explanations for the ranking results.

26 September 2024 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
SZShuai Zhao
Shuai Zhao
LZLinchao Zhu
Linchao Zhu
XWXiaohan Wang
Xiaohan Wang
YYYi Yang
Yi Yang

28 May 2021
Verified
Conference Paper
Closed access
YLYinggang Li
Yinggang Li
JHJunwei Hu
Junwei Hu
JSJifeng Sun
Jifeng Sun
SZShuai Zhao
Shuai Zhao
QZQi Zhang
Qi Zhang
YLYibin Lin
Yibin Lin
Self-supervised learning can be adopted to mine deep semantic information of visual data without a large number of human-annotated supervision by using a pretext task to pretrain a model. In this study, we proposed a novel self-supervised learning paradigm, namely multi-task self-supervised (MTSS) representation learning. Unlike existing self-supervised learning methods, which pretrain neural networks on the pretext task and then fine-tune the parameters of neural networks on the downstream task, in our scheme, downstream and pretext tasks are considered primary and auxiliary tasks, respectively, and are trained simultaneously. Our method involves maximizing the similarity of two augmented views of an image as an auxiliary task and using a multi-task network to train the primary task alongside the auxiliary task. We evaluated the proposed method on standard datasets and backbones through a rigorous experimental procedure. Experimental results revealed that proposed MTSS can achieve better performance and robustness than other self-supervised learning methods on multiple image classification data sets without using negative sample pairs and large batches. This simple yet effective method can inspire people to rethink self-supervised learning.

1 May 2021
Verified
Conference Paper
Closed access
QZQi Zhang
Qi Zhang
JSJifeng Sun
Jifeng Sun
YLYinggang Li
Yinggang Li
JHJunwei Hu
Junwei Hu
SZShuai Zhao
Shuai Zhao
YLYibin Lin
Yibin Lin
Recently, deep convolution neural networks achieve remarked performance in single image super-resolution(SISR) due to their strong feature representation ability. However, most existing SR methods mainly use standard convolution in each layer, neglecting to explore the various feature of spatial domain. Standard convolution focuses on spatial invariance by using the same convolution kernel in space. It deprives kernels of the ability to adapt the different spatial positions, hence hindering the diverse of feature. Later, a new convolution operator, Involution, was proposed to implement spatial-specific convolution. But it calculated the convolution kernel parameters of surrounding pixels only by utilizing the characteristics of the central pixel, which was unreasonable and limited its performance. To address these issues, in this paper, we propose a novel convolution named Revolution, which not only incorporates spatial-specific to capture different spatial information, but also considers the relevance of the pixels to determine convolution kernel parameters to further learn more realistic feature representations. On the other hand, the Revolution we proposed makes the number of parameters smaller. Experiments demonstrate that our method obtains obvious improvement in terms of indicators and visual effects.