Chengcheng Ma

Chengcheng Ma

Auto-generated profile, based on publicly available information

Chengcheng Ma

School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 February 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
CMChengcheng Ma
Chengcheng Ma
YLYang Liu
Yang Liu
JDJiankang Deng
Jiankang Deng
LXLingxi Xie
Lingxi Xie
WDWeiming Dong
Weiming Dong
CXChangsheng Xu
Changsheng Xu
Pretrained vision-language models (VLMs) such as CLIP have shown impressive generalization capability in downstream vision tasks with appropriate text prompts. Instead of designing prompts manually, Context Optimization (CoOp) has been recently proposed to learn continuous prompts using task-specific training data. Despite the performance improvements on downstream tasks, several studies have reported that CoOp suffers from the overfitting issue in two aspects: (i) the test accuracy on base classes first improves and then worsens during training;(ii) the test accuracy on novel classes keeps decreasing. However, none of the existing studies can understand and mitigate such overfitting problems. In this study, we first explore the cause of overfitting by analyzing the gradient flow. Comparative experiments reveal that CoOp favors generalizable and spurious features in the early and later training stages, respectively, leading to the non-overfitting and overfitting phenomena. Given those observations, we propose Subspace Prompt Tuning ( Sub PT) to project the gradients in back-propagation onto the low-rank subspace spanned by the early-stage gradient flow eigenvectors during the entire training process and successfully eliminate the overfitting problem. In addition, we equip CoOp with a Novel Feature Learner (NFL) to enhance the generalization ability of the learned prompts onto novel categories beyond the training set, needless of image training data. Extensive experiments on 11 classification datasets demonstrate that Sub PT+NFL consistently boost the performance of CoOp and outperform the state-of-the-art CoCoOp approach. Experiments on more challenging vision downstream tasks, including open-vocabulary object detection and zero-shot semantic segmentation, also verify the effectiveness of the proposed method. Codes can be found at https://tinyurl.com/mpe64f89.

25 April 2022 in Machine Intelligence Research
Verified
Journal Article
Closed access
PDF file
CMCheng-Cheng Ma
Cheng-Cheng Ma
BWBao-Yuan Wu
Bao-Yuan Wu
YFYan-Bo Fan
Yan-Bo Fan
YZYong Zhang
Yong Zhang
ZLZhi-Feng Li
Zhi-Feng Li

16 January 2025 in Machine Intelligence Research
Verified
Journal Article
Closed access
PDF file
CMChengcheng Ma
Chengcheng Ma
WDWeiming Dong
Weiming Dong
CXChangsheng Xu
Changsheng Xu