Siteng Huang

Siteng Huang

Auto-generated profile, based on publicly available information

Siteng Huang

Zhejiang University, Hangzhou, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 November 2019
Verified
Conference Paper
Open access
PDF file
SHSiteng Huang
Siteng Huang
DWDonglin Wang
Donglin Wang
XWXuehan Wu
Xuehan Wu
ATAo Tang
Ao Tang

18 March 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XLXuyang Liu
Xuyang Liu
TLTing Liu
Ting Liu
SHSiteng Huang
Siteng Huang
YXYi Xin
Yi Xin
YHYue Hu
Yue Hu
Honggang ChenHonggang Chen
Honggang Chen
Referring expression comprehension (REC) is a vision-language task to locate a target object in an image based on a language expression. Fully fine-tuning general-purpose pre-trained vision-language foundation models for REC yields impressive performance but becomes increasingly costly. Parameter-efficient transfer learning (PETL) methods have shown strong performance with fewer tunable parameters. However, directly applying PETL to REC faces two challenges: (1) insufficient multi-modal interaction between pre-trained vision-language foundation models, and (2) high GPU memory usage due to gradients passing through the heavy vision-language foundation models. To this end, we present M2IST: Multi-Modal Interactive Side-Tuning with M3ISAs: Mixture of Multi-Modal Interactive Side-Adapters. During fine-tuning, we fix the pre-trained uni-modal encoders and update M3ISAs to enable efficient vision-language alignment for REC. Empirical results reveal that M2IST achieves better performance-efficiency trade-off than full fine-tuning and other PETL methods, requiring only 2.11% tunable parameters, 39.61% GPU memory, and 63.46% training time while maintaining competitive performance. Our code is released at https://github.com/xuyang-liu16/M2IST.

28 October 2024
Verified
Conference Paper
Open access
CCCan Cui
Can Cui
SHSiteng Huang
Siteng Huang
WSWenxuan Song
Wenxuan Song
PDPengxiang Ding
Pengxiang Ding
MZMin Zhang
Min Zhang
Donglin WangDonglin Wang
Donglin Wang
To address the occlusion issues in person Re-Identification (ReID) tasks, many methods have been proposed to extract part features by introducing external spatial information. However, due to missing part appearance information caused by occlusion and noisy spatial information from external model, these purely vision-based approaches fail to correctly learn the features of human body parts from limited training data and struggle in accurately locating body parts, ultimately leading to misaligned part features. To tackle these challenges, we propose a Prompt-guided Feature Disentangling method (ProFD), which leverages the rich pre-trained knowledge in the textual modality facilitate model to generate well-aligned part features. ProFD first designs part-specific prompts and utilizes noisy segmentation mask to preliminarily align visual and textual embedding, enabling the textual prompts to have spatial awareness. Furthermore, to alleviate the noise from external masks, ProFD adopts a hybrid-attention decoder, ensuring spatial and semantic consistency during the decoding process to minimize noise impact. Additionally, to avoid catastrophic forgetting, we employ a self-distillation strategy, retaining pre-trained knowledge of CLIP to mitigate over-fitting. Evaluation results on the Market1501, DukeMTMC-ReID, Occluded-Duke, Occluded-ReID, and P-DukeMTMC datasets demonstrate that ProFD achieves state-of-the-art results.

9 March 2026 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
HZHaoyu Zhao
Haoyu Zhao
CZCheng Zeng
Cheng Zeng
LZLinghao Zhuang
Linghao Zhuang
YZYaxi Zhao
Yaxi Zhao
SXShengke Xue
Shengke Xue
ZHzou hua
zou hua
The scalability of robotic learning is fundamentally bottlenecked by the significant cost and labor of real-world data collection. While simulated data offers a scalable alternative, it often fails to generalize to the real world due to significant gaps in visual appearance, physical properties, and object interactions. To address this, we propose RoboSimGS, a novel Real2Sim2Real framework that converts multi-view real-world images into high-fidelity, and physically interactive simulation environments for robotic manipulation. Our approach reconstructs scenes using a hybrid representation: 3D Gaussian Splatting (3DGS) captures the photorealistic appearance of the environment, while mesh primitives for interactive objects ensure accurate physics simulation. Crucially, our framework leverages a Multi-modal Large Language Model (MLLM) to automate the creation of physically plausible, articulated assets from visual data. The MLLM analyzes visual data to infer not only physical properties (e.g., density, stiffness) but also kinematic structures (e.g., hinges, sliding rails) of objects. We demonstrate that policies trained entirely on data generated by RoboSimGS achieve successful zero-shot Sim2Real transfer across a diverse set of real-world manipulation tasks. Furthermore, data from RoboSimGS significantly enhances the performance and generalization capabilities of SOTA methods. Our results validate RoboSimGS as a powerful and scalable solution for bridging the Sim2Real gap.

12 June 2023
Verified
Conference Paper
Open access
SHSiteng Huang
Siteng Huang
QWQiyao Wei
Qiyao Wei
Donglin WangDonglin Wang
Donglin Wang
Compositional zero-shot learning (CZSL) refers to recognizing unseen compositions of known visual primitives, which is an essential ability for artificial intelligence systems to learn and understand the world. While considerable progress has been made on existing benchmarks, we suspect whether popular CZSL methods can address the challenges of few-shot and few referential compositions, which is common when learning in real-world unseen environments. To this end, we study the challenging reference-limited compositional zero-shot learning (RL-CZSL) problem in this paper, i.e., given limited seen compositions that contain only a few samples as reference, unseen compositions of observed primitives should be identified. We propose a novel Meta Compositional Graph Learner (MetaCGL) that can efficiently learn the compositionality from insufficient referential information and generalize to unseen compositions. Besides, we build a benchmark with two new large-scale datasets that consist of natural images with diverse compositional labels, providing more realistic environments for RL-CZSL. Extensive experiments in the benchmarks show that our method achieves state-of-the-art performance in recognizing unseen compositions when reference is limited for compositional learning.

1 January 2026 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
TLTing Liu
Ting Liu
XLXuyang Liu
Xuyang Liu
LSLiangtao Shi
Liangtao Shi
ZXZunnan Xu
Zunnan Xu
YHYue Hu
Yue Hu
Donglin WangDonglin Wang
Donglin Wang