Jiawei Liu

Jiawei Liu

Auto-generated profile, based on publicly available information

Jiawei Liu

University of Science and Technology of China

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 December 2023 in IEEE Transactions on Information Forensics and Security
Verified
Journal Article
Closed access
JLJiawei Liu
Jiawei Liu
JXJingyi Xie
Jingyi Xie
YWYang Wang
Yang Wang
ZZZheng-Jun Zha
Zheng-Jun Zha
Although existing face forgery detection methods achieve satisfactory performance under closed within-dataset scenario where training and testing sets are created by the same manipulation technique, they are vulnerable to samples created by unseen manipulation techniques under cross-dataset scenario. To solve this problem, in this work, we propose a novel adaptive texture and spectral clue mining (ATSC) approach for generalizable face forgery detection. It adaptively adjusts the parameters depended on input images to mine specific intrinsic forgery clues on both spatial and frequency domains. Specifically, ATSC customizes a Texture Clue Mining Module and a Spectrum Clue Selecting Module. The former module exploits instance-aware dynamic convolution on spatial domain to dynamically assemble multiple parallel convolutional kernels based on the learned image-dependent attention maps for effectively capturing subtle texture artifacts on spatial domain. A customized attention loss is also applied to the attention maps as supervision to precisely localize forgery artifacts whilst retain useful background information from suspicious and non-suspicious regions, which drives the module to explore all potential crucial clues and learning robust texture-related forgery feature. Moreover, the latter module applies adaptive frequency filtering mechanism on DCT-based frequency signals, which selects frequency information of interest to capture refined spectrum clues on frequency domain in an input-adaptive manner. Equipped with the above two modules, ATSC can learn more generalizable forgery features for face forgery detection. Extensive experimental results demonstrate the superior generalization ability of the proposed ATSC over various state-of-the-art methods on the challenging benchmarks.

26 October 2023
Verified
Conference Paper
Open access
FZFanrui Zhang
Fanrui Zhang
JLJiawei Liu
Jiawei Liu
QZQiang Zhang
Qiang Zhang
ESEsther Sun
Esther Sun
JXJingyi Xie
Jingyi Xie
ZZZheng-Jun Zha
Zheng-Jun Zha
Recently, falsified claims incorporating both text and images have been disseminated more effectively than those containing text alone, raising significant concerns for multi-modal fact verification. Existing research makes contributions to multi-modal feature extraction and interaction, but fails to fully utilize and enhance the valuable and intricate semantic relationships between distinct features. Moreover, most detectors merely provide a single outcome judgment and lack an inference process or explanation. Taking these factors into account, we propose a novel Explainable and Context-Enhanced Network (ECENet) for multi-modal fact verification, making the first attempt to integrate multi-clue feature extraction, multi-level feature reasoning, and justification (explanation) generation within a unified framework. Specifically, we propose an Improved Coarse- and Fine-grained Attention Network, equipped with two types of level-grained attention mechanisms, to facilitate a comprehensive understanding of contextual information. Furthermore, we propose a novel justification generation module via deep reinforcement learning that does not require additional labels. In this module, a sentence extractor agent measures the importance between the query claim and all document sentences at each time step, selecting a suitable amount of high-scoring sentences to be rewritten as the explanation of the model. Extensive experiments demonstrate the effectiveness of the proposed method.

5 May 2023 in Pain
Verified
Journal Article
Open access
JFJihong Fang
Jihong Fang
WWWei Wu
Wei Wu
JLJiawei Liu
Jiawei Liu
SZSicheng Zhang
Sicheng Zhang
Current automated pain assessment methods only focus on infants or youth. They are less practical because the children who suffer from postoperative pain in clinical scenarios are in a wider range of ages. In this article, we present a large-scale Clinical Pain Expression of Children (CPEC) dataset for postoperative pain assessment in children. It contains 4104 preoperative videos and 4865 postoperative videos of 4104 children (from 0 to 14 years of age), which are collected from January 2020 to December 2020 in Anhui Provincial Children's Hospital. Moreover, inspired by the dramatic successful applications of deep learning in medical image analysis and emotion recognition, we develop a novel deep learning–based framework to automatically assess postoperative pain according to the facial expression of children, namely Children Pain Assessment Neural Network (CPANN). We train and evaluate the CPANN with the CPEC dataset. The performance of the framework is measured by accuracy and macro-F1 score metrics. The CPANN achieves 82.1% accuracy and 73.9% macro-F1 score on the testing set of CPEC. The CPANN is faster, more convenient, and more objective compared with using pain scales according to the specific type of pain or children's condition. This study demonstrates the effectiveness of deep learning–based method for automated pain assessment in children.

11 February 2025 in IEEE Transactions on Information Forensics and Security
Verified
Journal Article
Closed access
YQYuxuan Qiu
Yuxuan Qiu
LWLiyang Wang
Liyang Wang
WSWei Song
Wei Song
JLJiawei Liu
Jiawei Liu
ZSZhiping Shi
Zhiping Shi
Na JiangNa Jiang
Na Jiang
Visible-infrared person re-identification (VI-ReID) is a critical cross-modality fine-grained classification task with significant implications for public safety and security applications. Existing VI-ReID methods primarily focus on extracting modality-invariant features for person retrieval. However, due to the inherent lack of texture information in infrared images, these modality-invariant features tend to emphasize global contexts. Consequently, individuals with similar silhouettes are often misidentified, posing potential risks to security systems and forensic investigations. To address this problem, this paper innovatively introduces natural language descriptions to learn the global-local contexts for VI-ReID. Specifically, we design a framework that jointly optimizes visible-infrared alignment plus (VIAP) and visual-textual reasoning (VTR), and introduces local-global joint measure (LJM) to enhance the metric, while proposing a human-LLM collaborative approach to incorporate textual descriptions into existing cross-modal person re-identification datasets. VIAP achieves cross-modal alignment between RGB and IR. It can explicitly utilize designed frequency-aware modality alignment and relationship-reinforced fusion to explore the potential of local cues in global features and modality-invariant information. VTR proposes pooling selection and dual-level reasoning mechanisms to force the image encoder to pay attention to significant regions based on textual descriptions. LJM proposes introducing local feature distances into the measure stage metric to enhance the relevance of matching using fine-grained information. Extensive experimental results on the popular SYSU-MM01 and RegDB datasets show that the proposed method significantly outperforms state-of-the-art approaches. The dataset is publicly available at https://github.com/qyx596/vireid-caption.

13 May 2024
Verified
Conference Paper
Open access
FZFanrui Zhang
Fanrui Zhang
JLJiawei Liu
Jiawei Liu
JXJingyi Xie
Jingyi Xie
QZQiang Zhang
Qiang Zhang
YXYongchao Xu
Yongchao Xu
ZZZheng-Jun Zha
Zheng-Jun Zha
Recently, misinformation incorporating both texts and images has been disseminated more effectively than those containing text alone on social media, raising significant concerns for multi-modal fact-checking. Existing research makes contributions to multi-modal feature extraction and interaction, but fails to fully enhance the valuable semantic representations or excavate the intricate entity information. Besides, existing multi-modal fact-checking datasets are primarily focused on English and merely concentrate on a single type of misinformation, thereby neglecting a comprehensive summary and coverage of various types of misinformation. Taking these factors into account, we construct the first large-scale Chinese Multi-modal Fact-Checking (CMFC) dataset which encompasses 46,000 claims. The CMFC covers all types of misinformation for fact-checking and is divided into two sub-datasets, Collected Chinese Multi-modal Fact-Checking (CCMF) and Synthetic Chinese Multi-modal Fact-Checking (SCMF). To establish baseline performance, we propose a novel Entity-enhanced and Stance Checking Network (ESCNet), which includes Multi-modal Feature Extraction Module, Stance Transformer, and Entity-enhanced Encoder. The ESCNet jointly models stance semantic reasoning features and knowledge-enhanced entity pair features, in order to simultaneously learn effective semantic-level and knowledge-level claim representations. Our work offers the first step and establishes a benchmark for evidence-based, multi-type, multi-modal fact-checking.

26 October 2023
Verified
Conference Paper
Open access
QZQiang Zhang
Qiang Zhang
JLJiawei Liu
Jiawei Liu
FZFanrui Zhang
Fanrui Zhang
JXJingyi Xie
Jingyi Xie
ZZZheng-Jun Zha
Zheng-Jun Zha
The explosion of multimodal fake news content on social media has sparked widespread concern. Existing multimodal fake news detection methods have made significant contributions to the development of this field, but fail to adequately exploit the potential semantic information of images and ignore the noise embedded in news entities, which severely limits the performance of the models. In this paper, we propose a novel Hierarchical Semantic Enhancement Network (HSEN) for multimodal fake news detection by learning text-related image semantic and precise news high-order knowledge semantic information. Specifically, to complement the image semantic information, HSEN utilizes textual entities as the prompt subject vocabulary and applies reinforcement learning to discover the optimal prompt format for generating image captions specific to the corresponding textual entities, which contain multi-level cross-modal correlation information. Moreover, HSEN extracts visual and textual entities from image and text, and identifies additional visual entities from image captions to extend image semantic knowledge. Based on that, HSEN exploits an adaptive hard attention mechanism to automatically select strongly related news entities and remove irrelevant noise entities to obtain precise high-order knowledge semantic information, while generating attention mask for guiding cross-modal knowledge interaction. Extensive experiments show that our method outperforms state-of-the-art methods.