Jiaming Zhou

Jiaming Zhou

Auto-generated profile, based on publicly available information

Jiaming Zhou

AI Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 August 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JZJiaming Zhou
Jiaming Zhou
KLKun-Yu Lin
Kun-Yu Lin
YQYu-Kun Qiu
Yu-Kun Qiu
WZWei-Shi Zheng
Wei-Shi Zheng
The long-term action in untrimmed video generally contains multiple sub-actions, among which various semantic patterns exist ( e.g. , the co-occurrence or sequentiality between sub-actions). These semantic patterns are temporally coarse, and correlated with multiple local contexts which encode the local temporal evolution of visual elements ( e.g. , hands, objects) in videos. The local contexts and semantic patterns form the inherent fine-to-coarse temporal structure of long-term actions, which is neglected by existing works. Accordingly, in this work we propose TwinFormer, which exploits a novel fine-to-coarse temporal modeling manner to uncover the temporal structure of long-term actions. The proposed TwinFormer consists of a pair of twin encoders with the same structural design, namely Localcontext Encoder and Semantic-pattern Encoder, and a Temporalbridged Attention to bridge the two twin encoders. The Localcontext Encoder aims to model the local contexts in the longterm action. And the Temporal-bridged Attention is designed to correlate the local contexts with semantic patterns. Furthermore, the Semantic-pattern Encoder reveals the temporal evolution of semantic patterns. Experimental results on three benchmarks demonstrate the effectiveness of the proposed model.

1 January 2026 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
KLKun-Yu Lin
Kun-Yu Lin
HDHenghui Ding
Henghui Ding
JDJia-Run Du
Jia-Run Du
JZJiaming Zhou
Jiaming Zhou
YPYi-Xing Peng
Yi-Xing Peng
WZWei-Shi Zheng
Wei-Shi Zheng
Inspired by the impressive success of image-text foundation models, recent works have proposed to adapt these foundation models to video data, leading to efficient and effective video models for open-vocabulary action recognition. However, through a comprehensive evaluation, our work finds that state-of the-art open-vocabulary action recognition models still struggle with generalization to video domains that they have not en countered. To address this limitation, we introduce generalizable open-vocabulary action recognition, which aims to develop action recognition models capable of generalizing to both novel action categories and unseen video domains. Our work contributes a novel model named XOV-Action to overcome two critical challenges: (1) understanding novel action concepts of open-set categories, and (2) mitigating the scenario discrepancy between training and test datasets. Specifically, XOV-Action first proposes to capture diverse action-related concepts by learning diversified elaboration representations, which enables better generalization to open-set action categories. Second, XOV-Action learns scene agnostic video representations to overcome the scene bias, which improves the generalization in unseen video domains. Addition ally, to evaluate models in generalizable open-vocabulary action recognition, we contribute a new cross-domain action benchmark named XOVABench, which covers multiple video domains with varying degrees of gaps and consists of both closed-set and open-set action categories. Extensive quantitative and qualitative experiments demonstrate that our proposed XOV-Action can effectively improve the action recognition performance for both closed-set and open-set categories across video domains.