Jian-Fang Hu

Prof. Jian-Fang Hu


Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 October 2024
Verified
Book Chapter
Closed access
PDF file
XLXiaotong Lin
Xiaotong Lin
TLTianming Liang
Tianming Liang
JLJianhuang Lai
Jianhuang Lai
Jian-Fang HuJian-Fang Hu
Jian-Fang Hu

29 August 2023 in Visual Intelligence
Verified
Journal Article
Open access
PDF file
JTJianwei Tang
Jianwei Tang
JWJieming Wang
Jieming Wang
Jian-Fang HuJian-Fang Hu
Jian-Fang Hu
Human motion prediction is a challenging task due to the diversity and randomness of future poses. Due to the inherent topology of pose data, most recent work has used graph convolution networks (GCNs) to accomplish the task of human motion prediction. However, the GCN-based method has a major shortcoming in that the modeling of temporal information is insufficient. In this paper, we propose a simple approach that combines recurrent neural networks (RNNs) and attention mechanism for motion prediction, which considers both spatial relations between different joints and temporal correlation. The query, key and value of the attention mechanism allow us to select the information we need for the subsequent prediction. To solve the difficult problem of RNN training, we utilize uncertainty and strong short-term constraints to optimize the training process. We evaluate our method on several standard benchmark datasets for human motion prediction, i.e., the Human3.6M dataset and the CMU MoCap dataset. The experimental results show that our approach outperforms previous approaches.

27 January 2025 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JMJingke Meng
Jingke Meng
HTHuilin Tian
Huilin Tian
GLGe Lin
Ge Lin
Jian-Fang HuJian-Fang Hu
Jian-Fang Hu
WZWei-Shi Zheng
Wei-Shi Zheng
Weakly supervised anomaly detection is to identify the time window when an anomaly event happened based on the video-level label indicating whether the video contains anomaly event or not. Recent efforts have focused on leveraging multi-modal data, specifically combining visual and audio information, to enhance detection accuracy. While some studies have explored intra-video separation techniques, the primary emphasis remains on distinguishing potentially anomalous events scoring highest from those scoring lowest as normal events. Nevertheless, challenges persist in delineating boundaries between normal and abnormal events, particularly when visual differences are subtle. Our proposed framework, called Audio-Visual Collaborative Learning (AVCL), addresses the challenge of ambiguity in weakly supervised anomaly detection. Our core idea centers around utilizing both audio track variations and the perceptual robustness of visual information to detect and differentiate challenging cases, which composed of two essential modules: the Audio-Visual collaborative Hard cases Separation (AVHS) module and the Multi-modal Mutual Learning (MML) module. The AVHS module aims to address the challenge of discerning visually ambiguous clips in anomaly videos, differentiating between normal and abnormal events. To further improve detection accuracy, we introduce the Multi-modal Mutual Learning (MML) module, and this module enables a process of mutual learning to facilitate the exchange of knowledge and expertise between the single-modal branch and the multi-modal branch. We demonstrate that the proposed approach achieves state-of-the-art detection performance on benchmarks of XD-Violence and CCTV-Fights$_{sub}$ datasets.

5 August 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
LLLinhui Li
Linhui Li
XLXiaotong Lin
Xiaotong Lin
YHYejia Huang
Yejia Huang
ZZZizhen Zhang
Zizhen Zhang
Jian-Fang HuJian-Fang Hu
Jian-Fang Hu
Pedestrian trajectory prediction is an essential task in real-world applications, aimed at predicting plausible future trajectories based on limited observations. In this work, we rethink the standard evaluation metric of the pedestrian trajectory prediction task: Minimum-of-N Average Displacement Error (MoN-ADE). As for multi-modal prediction models that generate multiple trajectories for each pedestrian, this metric typically evaluates the model by only considering the one that is closest to the ground-truth trajectory. However, such an evaluation protocol cannot comprehensively evaluate the predictive ability of the model, and potentially encourage models to generate high-variance and dispersed trajectory distributions. This is quite impractical especially for many real-world scenes like autonomous driving that require precise and convergent trajectory predictions. To address these limitations, we design a novel metric towards comprehensive evaluation in pedestrian trajectory prediction, which moves beyond the traditional reliance on the closest prediction. Specifically, we replace the Minimum-of-N strategy with an insightful Random-Sampling-K strategy to calculate the expectations of the minimum ADE and formulate a novel metric: Area Under the Curve (AUC). Furthermore, motivated by the proposed metric, we introduce a novel objective function named K-Ensemble Loss, which guides the state-of-the-art models to optimize the whole prediction distribution and reduce the uncertainty caused by the high-variance predictions. Extensive experiments on three real-world datasets demonstrate that the proposed metric and objective function are provided with significant effectiveness and flexibility.

10 September 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
JDJia-Run Du
Jia-Run Du
JFJia-Chang Feng
Jia-Chang Feng
KLKun-Yu Lin
Kun-Yu Lin
FHFa-Ting Hong
Fa-Ting Hong
ZQZhongang Qi
Zhongang Qi
WZWei-Shi Zheng
Wei-Shi Zheng
Weakly-Supervised Temporal Action Localization (WSTAL) aims to localize and classify action instances in long untrimmed videos with only video-level category labels as supervision. A critical challenge of WSTAL is the large gap between video-level supervision and unavailable snippet-level supervision. Prevailing methods typically assign pseudo labels to snippets, but these methods suffer from significant noise caused by the pseudo snippet-level labels. In this work, we address the WSTAL from a novel category exclusion perspective, which gradually enhances the snippet-level supervision to bridge the gap. Our proposed Progressive Complementary Learning (ProCL) is inspired by the fact that, video-level labels precisely indicate the categories that all snippets surely do not belong to, which is ignored by previous works. Accordingly, we first exclude these surely non-existent categories by the deterministic complementary learning. And then, we introduce the entropy-based pseudo complementary learning that is able to exclude more categories for snippets of less ambiguity. Furthermore, for the remaining ambiguous snippets, we attempt to reduce the ambiguity by distinguishing foreground actions from the background. Extensive experimental results show that our method achieves new state-of-the-art performance on THUMOS14, ActivityNet1.3, and MultiTHUMOS benchmarks.

17 February 2025 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
TLTianming Liang
Tianming Liang
LLLinhui Li
Linhui Li
Jian-Fang HuJian-Fang Hu
Jian-Fang Hu
XYXiangyang Yu
Xiangyang Yu
WZWei-Shi Zheng
Wei-Shi Zheng
JLJianhuang Lai
Jianhuang Lai
Video question answering (Video-QA) has emerged as a core task in the vision-language domain, which requires the models to understand a given video and answer textual questions related to the video. Compared to conventional image-language tasks, Video-QA is designed for improving the models' capacity of memorizing and integrating multi-frame temporal cues associated with the questions. While significant performance improvements have recently been witnessed on public benchmarks, in this work, we rethink whether these improvements truly stem from better understanding of video temporal context as expected. To this end, we accomplish a strong single-frame baseline model trained with knowledge distillation. With this model, we surprisingly find that visiting only one single frame, without incorporating multi-frame and temporal information, is sufficient to achieve state-of-the-art (SOTA) performance on multiple mainstream benchmarks. This finding reveals the prevalence of single-frame bias in current benchmarks for the first time. Around the single-frame bias, we conduct an in-depth analysis on multiple popular benchmarks, which demonstrate that: (i) merely relying on one frame is able to achieve comparable performance with SOTA temporal Video-QA models; (ii) simply ensembling the prediction scores of only 3 separate frames is able to surpass temporal SOTAs. Furthermore, we observe that most of the benchmarks are biased towards central segments, and even the latest benchmarks tailored for temporal reasoning still suffer from severe single-frame bias. In case study, we find two key properties of low-bias instances: the question emphasizes temporal dependency and contextual understanding, and the associated video content presents significant variability in scenes, actions or interactions. Through further analysis on compositional reasoning datasets, we find that constructing explicit object/event interactions upon videos to fill in well-designed temporal question templates can effectively reduce the single-frame bias during annotation. We hope our analysis helps facilitate future efforts in the field towards mitigating static bias and highlighting temporal reasoning.