Tong Zhang

Tong Zhang

Auto-generated profile, based on publicly available information

Tong Zhang

College of Computer Science and Technology, Zhejiang University, Hangzhou, China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 February 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
HFHao Fang
Hao Fang
TZTong Zhang
Tong Zhang
XZXiaofei Zhou
Xiaofei Zhou
XZXinxin Zhang
Xinxin Zhang
Recently, Transformer-based offline video instance segmentation (VIS) solutions have made significant progress by decomposing the whole task into global segmentation map generation and instance discrimination. We argue that the quality of video queries that represent all instances in a video clip is crucial for offline VIS methods. Existing methods typically interact video queries with dense spatio-temporal features, resulting in significant computational complexity and redundant information. Thus, we propose a novel video instance segmentation framework, LBVQ, dedicated to learning better video queries. Specifically, we first obtain the frame queries for each frame independently without any complex inter-frame spatial-temporal association operations. Secondly, we propose an adaptive query initialization module (AQI), which adaptively integrates frame queries to initialize video queries instead of traditional random initialization strategies. This initialization method preserves rich instance clues and accelerates the optimization of the whole model. Finally, to enhance the quality of video queries, we propose a query propagation module (QPM) that captures relevant instance information in frame queries frame by frame, greatly improving the model’s understanding of long videos. By learning higher quality video queries, LBVQ achieves the state-of-the-art on VIS benchmarks with a ResNet-50 backbone: 52.2 AP, 44.8 AP on YouTube-VIS 2019 & 2021. Moreover, LBVQ achieves 39.7 AP on YouTube-VIS 2022 and 22.2 AP on OVIS, demonstrating superior potential for long videos. To further improve the quality of segmentation masks, a large-scale pretrained SAM is employed to refine the segmentation results. Code is available at https://github.com/fanghaook/LBVQ.

24 November 2021 in Environmental Science and Pollution Research
Verified
Journal Article
Closed access
PDF file
TZTong Zhang
Tong Zhang
XZXuming Zhuang
Xuming Zhuang
SAShakeel Ahmad
Shakeel Ahmad
TLTaeho Lee
Taeho Lee
CCChengbo Cao
Chengbo Cao
Shou-Qing NiShou-Qing Ni
Shou-Qing Ni

11 August 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
TZTong Zhang
Tong Zhang
Xiankai LuXiankai Lu
Xiankai Lu
HZHao Zhang
Hao Zhang
XNXiushan Nie
Xiushan Nie
YYYilong Yin
Yilong Yin
JSJianbing Shen
Jianbing Shen
Video Language Grounding is one of the most challenging cross-modal video understanding tasks. This task aims to localize a target moment semantically corresponding to a given language query in an untrimmed video. Many existing VLG methods rely on the proposal-based framework, despite the dominant performance achieved, they usually focus on interacting a few internal frames with the query to score segment proposals, trapping in the long-range dependencies when the proposal feature is limited. Meanwhile, adjacent proposals share similar visual semantics, making VLG models hard to align the accurate semantics of video-query contents and degenerating the ranking performance. To remedy the above limitations, we propose VLG-CRF by introducing the conditional random fields (CRFs) to handle the discrete yet indistinguishable proposals. Specifically, VLG-CRF consists of two cascade CRF-based modules. The AttentiveCRFs is developed for multi-modal feature fusion to better integrate temporal and semantic relation between modalities. We also devise a new variant of ConvCRFs to capture the relation of discrete segments and rectify the predicting scores to make relatively high prediction scores clustered in a range. Experiments on three benchmark datasets, i.e. , Charades-STA, ActivityNet-Caption, and TACoS, show the superiority of our method and the state-of-the-art performance is achieved.

17 April 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
TZTong Zhang
Tong Zhang
HFHao Fang
Hao Fang
HZHao Zhang
Hao Zhang
JGJialin Gao
Jialin Gao
Xiankai LuXiankai Lu
Xiankai Lu
YYYilong Yin
Yilong Yin
Spatio-temporal video grounding (STVG) aims to localize a spatio-temporal tube, including temporal boundaries and object bounding boxes, that semantically corresponds to a given language description in an untrimmed video. The existing onestage solutions in this task face two significant challenges, namely, vision-text semantic misalignment and spatial mislocalization, which limit their performance in grounding. These two limitations are mainly caused by neglect of fine-grained alignment in crossmodality fusion and the reliance on a text-agnostic query in sequentially spatial localization. To address these issues, we propose an effective model with a newly designed Feature Semantic Matching (FSM) module based on a Transformer architecture to address the above issues. Our method introduces a crossmodal feature matching module to achieve multi-granularity alignment between video and text while preventing the weakening of important features during the feature fusion stage. Additionally, we design a query-modulated matching module to facilitate text-relevant tube construction by multiple query generation and tubulet sequence matching. To ensure the quality of tube construction, we employ a novel mismatching rectify contrastive loss to rectify the mismatching between the learnable query and the objects corresponding to the text descriptions by restricting the generated spatial query. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods on two challenging STVG benchmarks.

8 May 2026 in Machine Intelligence Research
Verified
Journal Article
Closed access
PDF file
YJYi Jiang
Yi Jiang
OMOubo Ma
Oubo Ma
YYYong Yang
Yong Yang
TZTong Zhang
Tong Zhang
SJShouling Ji
Shouling Ji

24 June 2026 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
XWXin Wang
Xin Wang
TZTong Zhang
Tong Zhang
YGYongshun Gong
Yongshun Gong
JGJialin Gao
Jialin Gao
YXYanyu Xu
Yanyu Xu
CZChengqi Zhang
Chengqi Zhang
Spatio-temporal video grounding (STVG) aims to precisely locate a spatio-temporal tube in an untrimmed video corresponding to a given language description. Many existing methods decouple spatial and temporal grounding as separate tasks, missing the strong interdependencies between the two, which are crucial for accurately aligning spatial regions (such as objects) with their motion over time. Thus, to enhance spatio-temporal associations, we introduce a new Prior-Driven Transformer Network (PDTNet) with predicted temporal boundaries as priors to guide object bounding boxes for improved spatial grounding over time. Firstly, PDTNet employs a temporal prior, termed reference query, to enhance discriminability between language-related and language-irrelevant visual content, improving temporal boundary localization. Further, the context within predicted temporal boundaries serves as another prior knowledge to modulate spatial features. We also introduce a prediction-aware Gaussian prior to precise object localization. This ensures consistent tube construction and accurate object localization. Extensive experiments on STVG benchmarks validate the effectiveness of PDTNet. Code is available at https://github.com/tongzhang111/PDTNet .