Fan Zhang

Fan Zhang

Auto-generated profile, based on publicly available information

Fan Zhang

Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 October 2023
Verified
Conference Paper
Open access
ZCZebang Cheng
Zebang Cheng
YLYuxiang Lin
Yuxiang Lin
ZCZhaoru Chen
Zhaoru Chen
XLXiang Li
Xiang Li
SMShuyi Mao
Shuyi Mao
Xiaojiang PengXiaojiang Peng
Xiaojiang Peng
The Multimodal Emotion Recognition (MER 2023) challenge aims to recognize emotion with audio, language, and visual signals, facilitating innovative technologies of affective computing. This paper presents our submission approach on the Semi-Supervised Learning Sub-Challenge (MER-SEMI). First, with large-scale unlabeled emotional videos, we train both image-based and video-based Masked Autoencoders to extract visual features, which termed as expression MAE (expMAE) for simplicity. The expMAE features are found to be largely complementary with other official baseline features. Second, since there is only a few labeled data, we use a classifier to generate pseudo labels for unlabeled videos which have high confidence for a certain category. In addition, we also explore several advanced large models for cross-feature extraction like CLIP, and apply factorized bilinear pooling (FBP) for multimodal feature fusion. Our methods finally achieved 88.55% in F1 score on MER-SEMI, ranking second place among all participating teams.

27 January 2025 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
SMShuyi Mao
Shuyi Mao
XLXinpeng Li
Xinpeng Li
FZFan Zhang
Fan Zhang
Xiaojiang PengXiaojiang Peng
Xiaojiang Peng
YYYang Yang
Yang Yang
Label biases in facial expression recognition (FER) datasets, caused by annotators' subjectivity, pose challenges in improving the performance of target datasets when auxiliary labeled data are used. Moreover, training with multiple datasets can lead to visible degradations in the target dataset. To address these issues, we propose a novel framework called the AU-aware Vision Transformer (AU-ViT), which leverages unified action unit (AU) information and discards expression annotations of auxiliary data. AU-ViT integrates an elaborately designed AU branch in the middle part of a master ViT to enhance representation learning during training. Through qualitative and quantitative analyses, we demonstrate that AU-ViT effectively captures expression regions and is robust to real-world occlusions. Additionally, we observe that AU-ViT also yields performance improvements on the target dataset, even without auxiliary data, by utilizing pseudo AU labels. Our AU-ViT achieves performances superior to, or comparable to, that of the state-of-the-art methods on FERPlus, RAFDB, AffectNet, LSD and the other three occlusion test datasets.

11 June 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
FZFan Zhang
Fan Zhang
HZHang Zhou
Hang Zhou
XHXian-Sheng Hua
Xian-Sheng Hua
CCChong Chen
Chong Chen
XLXiao Luo
Xiao Luo
With the emergence of AI generated content, cross-modal retrieval of 2D and 3D data has obtained increasing research attention. In practical applications, massive amounts of 2D and 3D data need expensive annotation, which would make labels scarce. Even worse, complicated heterogeneous relationships between 2D and 3D data make the problem more challenging. In this research, we study the problem of semi-supervised 2D and 3D cross-modal retrieval and provide a novel method named H ierarchical Alignment with Ambigu o us P seudo-lab e ling (HOPE) for this problem. The core of HOPE is to align two modalities in the common space from a hierarchical perspective. Specifically, HOPE not only enforces each sample to approach its respective modality-invariant anchors from an individual view, but also measures both prototypes and distribution for both modalities for discrepancy reduction from a group view. To handle label scarcity with limited error accumulation, HOPE employs two branches of perturbed networks to generate ambiguous candidates, which guides the cross-branch supervision using a margin-based ranking objective. In addition, we retrieve reliable unlabeled samples for each anchor with curriculum learning and class balance, which are added into labeled datasets to clear ambiguity. Extensive experiments on various benchmark datasets validate the superiority of the proposed HOPE.

25 July 2025 in Computer Vision and Image Understanding
Verified
Journal Article
Closed access
FZFan Zhang
Fan Zhang
ZCZhi-Qi Cheng
Zhi-Qi Cheng
JZJian Zhao
Jian Zhao
Xiaojiang PengXiaojiang Peng
Xiaojiang Peng
XLXuelong Li
Xuelong Li

4 June 2024 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
FZFan Zhang
Fan Zhang
CCChong Chen
Chong Chen
XHXian-Sheng Hua
Xian-Sheng Hua
XLXiao Luo
Xiao Luo
Hashing has received significant interest in large-scale data retrieval due to its outstanding computational efficiency. Of late, numerous deep hashing approaches have emerged, which have obtained impressive performance. However, these approaches can contain ethical risks during image retrieval. To address this, we are the first to study the problem of group fairness within learning to hash and introduce a novel method termed Fairness-aware Hashing with Mixture of Experts (FATE). Specifically, FATE leverages the mixture-of-experts framework as the hashing network, where each expert contributes knowledge from an individual viewpoint, followed by aggregation using the gating mechanism. This strongly enhances the model capability, facilitating the generation of both discriminative and unbiased binary descriptors. We also incorporate fairness-aware contrastive learning, combining sensitive labels with feature similarities to ensure unbiased hash code learning. Furthermore, an adversarial learning objective condition on both deep features and hash codes is employed to further eliminate group biases. Extensive experiments on several benchmark datasets validate the superiority of the proposed FATE compared with various state-of-the-art approaches.

26 November 2025 in Expert Systems with Applications
Verified
Journal Article
Closed access
FZFan Zhang
Fan Zhang
SMShuyi Mao
Shuyi Mao
QLQing Li
Qing Li
Xiaojiang PengXiaojiang Peng
Xiaojiang Peng