Chen Jiang

Chen Jiang

Auto-generated profile, based on publicly available information

Chen Jiang

Ant Group Company Ltd., Hangzhou, China

Research Focus & Metrics

Academic Career Length

14Years (2013-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 October 2023
Verified
Conference Paper
Open access
CJChen Jiang
Chen Jiang
HLHong Liu
Hong Liu
XYXuzheng Yu
Xuzheng Yu
QWQing Wang
Qing Wang
YCYuan Cheng
Yuan Cheng
YQYuan Qi
Yuan Qi
In recent years, the explosion of web videos makes text-video retrieval increasingly essential and popular for video filtering, recommendation, and search. Text-video retrieval aims to rank relevant text/video higher than irrelevant ones. The core of this task is to precisely measure the cross-modal similarity between texts and videos. Recently, contrastive learning methods have shown promising results for text-video retrieval, most of which focus on the construction of positive and negative pairs to learn text and video representations. Nevertheless, they do not pay enough attention to hard negative pairs and lack the ability to model different levels of semantic similarity. To address these two issues, this paper improves contrastive learning using two novel techniques. First, to exploit hard examples for robust discriminative power, we propose a novel Dual-Modal Attention-Enhanced Module (DMAE) to mine hard negative pairs from textual and visual clues. By further introducing a Negative-aware InfoNCE (NegNCE) loss, we are able to adaptively identify all these hard negatives and explicitly highlight their impacts in the training loss. Second, our work argues that triplet samples can better model fine-grained semantic similarity compared to pairwise samples. We thereby present a new Triplet Partial Margin Contrastive Learning (TPM-CL) module to construct partial order triplet samples by automatically generating fine-grained hard negatives for matched text-video pairs. The proposed TPM-CL designs an adaptive token masking strategy with cross-modal interaction to model subtle semantic differences. Extensive experiments demonstrate that the proposed approach outperforms existing methods on four widely-used text-video retrieval datasets, including MSR-VTT, MSVD, DiDeMo and ActivityNet.

1 November 2013
Verified
Conference Paper
Closed access
YJYu Jun
Yu Jun
ALAljun Li
Aljun Li
FHFang Hu
Fang Hu
QFQiang Fang
Qiang Fang
CJChen Jiang
Chen Jiang
ZWZengfu Wang
Zengfu Wang
In the framework of intelligent aided language learning, a real-time data-driven 3D visual pronunciation system of Chinese IPA is proposed. First, a high quality articulatory speech corpus including speech and 3D articulatory data of lips, tongue and jaw movements is collected through Electro-Magnetic Articulograph; second, the 3D articulatory modeling including shape design and motion synthesis is conducted. The articulatory shape is obtained by designing a precise 3D facial model including internal and external articulators. The articulatory motion synthesis is obtained combining parameterized model and anatomical model. The system can thus illustrate the slight differences among phonemes by synthesizing both internal and external articulatory movements. The perceptual evaluation shows the suitability of the system for instructing language learners to articulate.

20 February 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XYXuzheng Yu
Xuzheng Yu
CJChen Jiang
Chen Jiang
XDXingning Dong
Xingning Dong
TGTian Gan
Tian Gan
MYMing Yang
Ming Yang
QGQingpei Guo
Qingpei Guo
The user base of short video apps has experienced unprecedented growth in recent years, resulting in a significant demand for video content analysis. In particular, text-video retrieval, which aims to find the top matching videos given text descriptions from a vast video corpus, is an essential function, the primary challenge of which is to bridge the modality gap. Nevertheless, most existing approaches treat texts merely as discrete tokens and neglect their syntax structures. Moreover, the abundant spatial and temporal clues in videos are often under-utilized due to the lack of interaction with text. To address these issues, we argue that using texts as guidance to focus on relevant temporal frames and spatial regions within videos is beneficial. In this paper, we propose a novel Syntax-Hierarchy-Enhanced text-video retrieval method (SHE-Net) that exploits the inherent semantic and syntax hierarchy of texts to bridge the modality gap from two perspectives. First, to facilitate a more fine-grained integration of visual content, we employ the text syntax hierarchy, which reveals the grammatical structure of text descriptions, to guide the visual representations. Second, to further enhance the multi-modal interaction and alignment, we also utilize the syntax hierarchy to guide the similarity calculation. We evaluated our method on four public text-video retrieval datasets of MSR-VTT, MSVD, DiDeMo, and ActivityNet. The experimental results and ablation studies confirm the advantages of our proposed method.

1 July 2014
Verified
Conference Paper
Closed access
CLChangwei Luo
Changwei Luo
YJYu Jun
Yu Jun
CJChen Jiang
Chen Jiang
RLRui Li
Rui Li
ZWZengfu Wang
Zengfu Wang
Facial animation is useful in human-machine interaction, computer games and teleconferences. We propose a realtime performance-driven facial animation system for ordinary users. The system enables a user to animate an avatar by performing desired facial motions in front of a video camera. First, a constrained local model based approach is used to track facial features of a performer in the video. To increase the tracking accuracy, we propose an efficient method to build a user-specific local texture model. Next, a 3D blendshape face model is fitted to the tracked feature points. To improve the expressiveness of synthesized animations, facial expression recognition results and pre-recorded animation priors are incorporated into the fitting procedure. Finally, facial animations are created using blendshape interpolation. Experiments show that the synthetic facial motions are realistic and quite similar to the facial actions of the performer. By using an ordinary camera, our system provides the user complete control over the generated facial animations.

16 June 2022 in arXiv
Verified
Preprint
Open access
PDF file
SHSifeng He
Sifeng He
Xudong YangXudong Yang
Xudong Yang
CJChen Jiang
Chen Jiang
GLGang Liang
Gang Liang
WZWei Zhang
Wei Zhang
Lei YangLei Yang
Lei Yang
In this paper, we introduce VCSL (Video Copy Segment Localization), a new comprehensive segment-level annotated video copy dataset. Compared with existing copy detection datasets restricted by either video-level annotation or small-scale, VCSL not only has two orders of magnitude more segment-level labelled data, with 160k realistic video copy pairs containing more than 280k localized copied segment pairs, but also covers a variety of video categories and a wide range of video duration. All the copied segments inside each collected video pair are manually extracted and accompanied by precisely annotated starting and ending timestamps. Alongside the dataset, we also propose a novel evaluation protocol that better measures the prediction accuracy of copy overlapping segments between a video pair and shows improved adaptability in different scenarios. By benchmarking several baseline and state-of-the-art segment-level video copy detection methods with the proposed dataset and evaluation metric, we provide a comprehensive analysis that uncovers the strengths and weaknesses of current approaches, hoping to open up promising directions for future works. The VCSL dataset, metric and benchmark codes are all publicly available at https://github.com/alipay/VCSL.