Lin Li

Lin Li

Auto-generated profile, based on publicly available information

Lin Li

AI Chip Center for Emerging Smart Systems, Hong Kong, China and The Hong Kong University of Science and Technology, Hong Kong, China

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 April 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
LLLin Li
Lin Li
JXJun Xiao
Jun Xiao
HSHanrong Shi
Hanrong Shi
HZHanwang Zhang
Hanwang Zhang
YYYi Yang
Yi Yang
LCLong Chen
Long Chen
Nearly all existing scene graph generation (SGG) models have overlooked the ground-truth annotation qualities of mainstream SGG datasets, i.e., they assume: 1) all the manually annotated positive samples are equally correct; 2) all the un-annotated negative samples are absolutely background. In this article, we argue that neither of the assumptions applies to SGG: there are numerous “noisy” ground-truth predicate labels that break these two assumptions and harm the training of unbiased SGG models. To this end, we propose a novel N o I sy label C orr E ction and S ample T raining strategy for SGG: NICEST , which rules out these noisy label issues by generating high-quality samples and designing an effective training strategy. Specifically, it consists of: 1) NICE : it detects noisy samples and then reassigns higher-quality soft predicate labels to them. To achieve this goal, NICE contains three main steps: negative Noisy Sample Detection (Neg-NSD), positive NSD (Pos-NSD), and Noisy Sample Correction (NSC). First, in Neg-NSD, it is treated as an out-of-distribution detection problem, and the pseudo labels are assigned to all detected noisy negative samples. Then, in Pos-NSD, we use a density-based clustering algorithm to detect noisy positive samples. Lastly, in NSC, we use weighted KNN to reassign more robust soft predicate labels rather than hard labels to all noisy positive samples. 2) NIST : it is a multi-teacher knowledge distillation based training strategy, which enables the model to learn unbiased fusion knowledge. A dynamic trade-off weighting strategy in NIST is designed to penalize the bias of different teachers. Due to the model-agnostic nature of both NICE and NIST, NICEST can be seamlessly incorporated into any SGG architecture to boost its performance on different predicate categories. In addition, to better assess the generalization ability of SGG models, we propose a new benchmark, VG-OOD , by reorganizing the prevalent VG dataset. This reorganization deliberately makes the predicate distributions between the training and test sets as different as possible for each subject-object category pair. This new benchmark helps disentangle the influence of subject-object category biases. Extensive ablations and results on different backbones and tasks have attested to the effectiveness and generalization ability of each component of NICEST.

2 June 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
LLLin Li
Lin Li
JXJun Xiao
Jun Xiao
HSHanrong Shi
Hanrong Shi
WWWenxiao Wang
Wenxiao Wang
JSJian Shao
Jian Shao
LCLong Chen
Long Chen
The Scene Graph Generation (SGG) task aims to detect all the objects and their pairwise visual relationships in a given image. Although SGG has achieved remarkable progress over the last few years, almost all existing SGG models follow the same training paradigm: they treat both object and predicate classification in SGG as a single-label classification problem, and the ground-truths are one-hot target labels. However, this prevalent training paradigm has overlooked two characteristics of current SGG datasets: 1) For positive samples, some specific subject-object instances may have multiple reasonable predicates. 2) For negative samples, there are numerous missing annotations. Regardless of the two characteristics, SGG models are easy to be confused and make wrong predictions. To this end, we propose a novel model-agnostic Label Semantic Knowledge Distillation (LS-KD) for unbiased SGG. Specifically, LS-KD dynamically generates a “soft" label for each subject-object instance by fusing a predicted Label Semantic Distribution (LSD) with its original one-hot target label. LSD reflects the correlations between this instance and multiple predicate categories. Meanwhile, we propose two different strategies to predict LSD: iterative self-KD and synchronous self-KD. Extensive ablations and results on three SGG tasks have attested to the superiority and generality of our proposed LS-KD, which can consistently achieve decent trade-off performance between different predicate categories.

22 September 2020 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JXJun Xiao
Jun Xiao
LLLin Li
Lin Li
DXDejing Xu
Dejing Xu
CLChengjiang Long
Chengjiang Long
JSJian Shao
Jian Shao
YZYueting Zhuang
Yueting Zhuang
We present a self-supervised spatiotemporal learning approach by exploring the temporal coherence of videos. The chronological order of shuffled clips from the video is used as the supervisory signal to guide the 3D Convolutional Neural Networks (CNNs) to learn meaningful visual knowledge. Unlike the existing approaches which use frames, we utilize dynamic video clips to reduce the uncertainty of order. We test three types of representative 3D CNNs, all of which benefit from the proposed approach. The learned 3D CNNs can be used either as a feature extractor or a pre-trained model for further fine-tuning on downstream tasks. We also propose two curriculum learning strategies to make the 3D CNNs easier to train and get the state-of-the-art results in nearest neighbor retrieval and action recognition tasks compared with other self-supervised learning methods. Meanwhile, it is further extended to the field of visual question answering application and has achieved promising results. Besides, comprehensive and extensive experimental results and analyses are provided for readers to better understand the video clip order we explore with self-supervised and curriculum learning for video application.

2 November 2022 in Expert Systems with Applications
Verified
Journal Article
Closed access
YYYonglin Yu
Yonglin Yu
HLHaifeng Li
Haifeng Li
HSHanrong Shi
Hanrong Shi
LLLin Li
Lin Li
JXJun Xiao
Jun Xiao

27 October 2025
Verified
Conference Paper
Open access
LLLin Li
Lin Li
GCGuikun Chen
Guikun Chen
ZWZhen Wang
Zhen Wang
JXJun Xiao
Jun Xiao
LCLong Chen
Long Chen

5 August 2024 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
HSHanrong Shi
Hanrong Shi
LLLin Li
Lin Li
JXJun Xiao
Jun Xiao
YZYueting Zhuang
Yueting Zhuang
LCLong Chen
Long Chen