Zilong Huang

Zilong Huang

Auto-generated profile, based on publicly available information

Zilong Huang

Tencent PCG, Shanghai, China

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 July 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
ZHZilong Huang
Zilong Huang
Xinggang WangXinggang Wang
Xinggang Wang
YWYunchao Wei
Yunchao Wei
LHLichao Huang
Lichao Huang
HSHumphrey Shi
Humphrey Shi
THThomas S. Huang
Thomas S. Huang
Contextual information is vital in visual understanding problems, such as semantic segmentation and object detection. We propose a criss-cross network (CCNet) for obtaining full-image contextual information in a very effective and efficient way. Concretely, for each pixel, a novel criss-cross attention module harvests the contextual information of all the pixels on its criss-cross path. By taking a further recurrent operation, each pixel can finally capture the full-image dependencies. Besides, a category consistent loss is proposed to enforce the criss-cross attention module to produce more discriminative features. Overall, CCNet is with the following merits: 1) GPU memory friendly. Compared with the non-local block, the proposed recurrent criss-cross attention module requires $11\times$ less GPU memory usage. 2) High computational efficiency. The recurrent criss-cross attention significantly reduces FLOPs by about 85 percent of the non-local block. 3) The state-of-the-art performance. We conduct extensive experiments on semantic segmentation benchmarks including Cityscapes, ADE20K, human parsing benchmark LIP, instance segmentation benchmark COCO, video segmentation benchmark CamVid. In particular, our CCNet achieves the mIoU scores of 81.9, 45.76 and 55.47 percent on the Cityscapes test set, the ADE20K validation set and the LIP validation set respectively, which are the new state-of-the-art results. The source codes are available at https://github.com/speedinghzl/CCNethttps://github.com/speedinghzl/CCNet .

1 March 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
ZHZilong Huang
Zilong Huang
YWYunchao Wei
Yunchao Wei
Xinggang WangXinggang Wang
Xinggang Wang
HSHumphrey Shi
Humphrey Shi
WLWenyu Liu
Wenyu Liu
THThomas S. Huang
Thomas S. Huang
Aggregating features in terms of different convolutional blocks or contextual embeddings has been proven to be an effective way to strengthen feature representations for semantic segmentation. However, most of the current popular network architectures tend to ignore the misalignment issues during the feature aggregation process caused by 1) step-by-step downsampling operations, and 2) indiscriminate contextual information fusion. In this paper, we explore the principles in addressing such feature misalignment issues and inventively propose Feature-Aligned Segmentation Networks (AlignSeg). AlignSeg consists of two primary modules, \ie, the Aligned Feature Aggregation (AlignFA) module and the Aligned Context Modeling (AlignCM) module. First, AlignFA adopts a simple learnable interpolation strategy to learn transformation offsets of pixels, which can effectively relieve the feature misalignment issue caused by multiresolution feature aggregation. Second, with the contextual embeddings in hand, AlignCM enables each pixel to choose private custom contextual information in an adaptive manner, making the contextual embeddings aligned better to provide appropriate guidance. We validate the effectiveness of our AlignSeg network with extensive experiments on Cityscapes and ADE20K, achieving new state-of-the-art mIoU scores of 82.6% and 45.95%, respectively. Our source code will be made available.

25 January 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
FYFukun Yin
Fukun Yin
ZHZilong Huang
Zilong Huang
BZBo Zhou
Bo Zhou
GLGuozhong Luo
Guozhong Luo
Gang YuGang Yu
Gang Yu
BFBin Fu
Bin Fu
The point cloud feature aggregation, which learns discriminative features from the disordered points, plays a key role for large-scale point cloud semantic segmentation. Most previous aggregation methods are based on sampling a representative point subset, i.e., by a carefully designed point density metric, facing expensive computation cost especially for large-scale point clouds. Even though speeding up the point sampling process is studied by several recent works, but the component points in the sampled subset are uncertain and may change randomly, thus leading to corrupted geometric structure and discarded edges for representing an object. Therefore, we propose the DCNet, which consists of a fast point random sampling based encoder-decoder structure and several fully connected layers for semantic segmentation. To overcome the key feature loss caused by random down-sampling, the DCNet develops two novel local feature aggregation schemes: Double attention and Consistent constraints, to learn features that are discriminative for the challenging scenarios as above. The former considers both topological and semantic similarity of neighboring points to generate attention features for discriminating classes with similar geometric structures. The latter develops class-consistent constraints between adjacent layers in the decoder stage, to guide each point to aggregate with high-level semantic features of points belonging to the same class from the previous layer, which is beneficial for distinguishing neighboring points of the same class on the boundary. We conduct experiments and compare the proposed DCNet with existing methods on two benchmarks S3DIS and Semantic3D. Experiments show that the mean Intersection-over-Union (mIoU) of our method outperforms state-of-the-art methods by 2-3%, based on the same fast random sampling, and is also comparable to latest sampling-slower but accuracy-higher methods. That is, our method achieves the optimal speed-accuracy trade-off in the field of point cloud segmentation.

1 November 2026 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
HYHaobo Yuan
Haobo Yuan
Xiangtai LiXiangtai Li
Xiangtai Li
TZTao Zhang
Tao Zhang
YSYueyi Sun
Yueyi Sun
ZHZilong Huang
Zilong Huang
MYMing Yang
Ming Yang
This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-modal large language models, which are often limited to specific modalities and tasks, Sa2VA supports a wide range of image and video tasks, including referring segmentation and conversation, with minimal single-stage instruction tuning. Sa2VA combines SAM-2, a foundation video segmentation model, with MLLM, advanced vision-language models, and unifies text, image, and video into a shared LLM token space. Using the LLM, Sa2VA generates instruction tokens that guide SAM-2 in producing precise masks, enabling a grounded, multi-modal understanding of both static and dynamic visual content. Additionally, we introduce Ref-SAV, an auto-labeled dataset containing over 72k object expressions in complex video scenes, designed to boost model performance. We also manually validate 2k video objects in the Ref-SAV dataset to benchmark referring video object segmentation in complex environments. Experiments show that Sa2VA achieves strong performance across multiple tasks, particularly in referring video object segmentation, highlighting its potential for complex real-world applications. In addition, Sa2VA can be easily extended to various MLLMs, including Qwen-VL and Intern-VL, which can be updated with a rapid process in current open-source MLLMs. Code and models have been provided to the community.

6 June 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
MLMingsheng Li
Mingsheng Li
LZLin Zhang
Lin Zhang
MZMingzhen Zhu
Mingzhen Zhu
ZHZilong Huang
Zilong Huang
Gang YuGang Yu
Gang Yu
BZBo Zhou
Bo Zhou
This paper studies the problem of pre-training for small models, which is essential for many mobile devices. Current state-of-the-art methods on this problem transfer the representational knowledge of a large network (as a Teacher) into a smaller model (as a Student) using self-supervised distillation, improving the performance of the small model on downstream tasks. However, existing approaches are insufficient in extracting the crucial knowledge that is useful for discerning categories in downstream tasks during the distillation process. In this paper, for the first time, we introduce language guidance to the distillation process and propose a new method named Language-Guided Distillation (LGD) system, which uses category names of the target downstream task to help refine the knowledge transferred between the teacher and student. To this end, we utilize a pre-trained text encoder to extract semantic embeddings from language and construct a textual semantic space called Textual Semantics Bank (TSB). Furthermore, we design a Language-Guided Knowledge Aggregation (LGKA) module to construct the visual semantic space, also named Visual Semantics Bank (VSB). The task-related knowledge is transferred by driving a student encoder to mimic the similarity score distribution inferred by a teacher over TSB and VSB. Compared with other small models obtained by either ImageNet pre-training or self-supervised distillation, experiment results show that the distilled lightweight model using the proposed LGD method presents state-of-the-art performance and is validated on various downstream tasks, including classification, detection, and segmentation. We have made the code available at https://github.com/mZhenz/LGD .

4 July 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
FYFukun Yin
Fukun Yin
ZHZilong Huang
Zilong Huang
BZBo Zhou
Bo Zhou
GLGuozhong Luo
Guozhong Luo
Gang YuGang Yu
Gang Yu
BFBin Fu
Bin Fu
Presents corrections to the paper, “DCNet: Large-Scale Point Cloud Semantic Segmentation With Discriminative and Efficient Feature Aggregation”.