Haipeng Jia

Haipeng Jia

Auto-generated profile, based on publicly available information

Haipeng Jia

University of Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 September 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
WWWenhao Wu
Wenhao Wu
XWXiaohan Wang
Xiaohan Wang
HLHaipeng Luo
Haipeng Luo
JWJingdong Wang
Jingdong Wang
YYYi Yang
Yi Yang
WOWanli Ouyang
Wanli Ouyang
Understanding videos, especially aligning them with textual data, presents a significant challenge in computer vision. The advent of vision-language models (VLMs) like CLIP has sparked interest in leveraging their capabilities for enhanced video understanding, showing marked advancements in both performance and efficiency. However, current methods often neglect vital user-generated metadata such as video titles. In this paper, we present Cap4Video++, a universal framework that leverages auxiliary captions to enrich video understanding. More recently, we witness the flourishing of large language models (LLMs) like ChatGPT. Cap4Video++ harnesses the synergy of vision-language models (VLMs) and large language models (LLMs) to generate video captions, utilized in three key phases: (i) Input stage employs Semantic Pair Sampling to extract beneficial samples from captions, aiding contrastive learning. (ii) Intermediate stage sees Video-Caption Cross-modal Interaction and Adaptive Caption Selection work together to bolster video and caption representations. (iii) Output stage introduces a Complementary Caption-Text Matching branch, enhancing the primary video branch by improving similarity calculations. Our comprehensive experiments on text-video retrieval and video action recognition across nine benchmarks clearly demonstrate Cap4Video++'s superiority over existing models, highlighting its effectiveness in utilizing automatically generated captions to advance video understanding.

28 January 2026
Verified
Conference Paper
Open access
XYXiaojian Yang
Xiaojian Yang
YNYuhui Ni
Yuhui Ni
FYFan Yuan
Fan Yuan
SLShengguo Li
Shengguo Li
DDDezun Dong
Dezun Dong
JLJie Liu
Jie Liu