Jiamin Zhuang

Jiamin Zhuang

Auto-generated profile, based on publicly available information

Jiamin Zhuang

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 May 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JZJiamin Zhuang
Jiamin Zhuang
JYJing Yu
Jing Yu
YDYang Ding
Yang Ding
XQXiangyan Qu
Xiangyan Qu
YHYue Hu
Yue Hu
Image-text retrieval requires the system to bridge the heterogenous gap between vision and language for accurate retrieval while keeping the network lightweight-enough for efficient retrieval. Existing trade-off solutions mainly study from the view of incorporating cross-modal interactions with the independent-embedding framework or leveraging stronger pre-trained encoders, which still demand time-consuming similarity measurement or heavyweight model structure in the retrieval stage. In this work, we propose an image-text alignment module SelfAlign on top of the independent-embedding framework, which improves the retrieval accuracy while maintains the retrieval efficiency without extra supervision. SelfAlign contains two collaborative sub-modules that force image-text alignment at both the concept level and context level by self-supervised contrastive learning. It doesn't require cross-modal embedding interactions during training while maintaining independent image and text encoders during retrieval. With comparable time cost, SelfAlign consistently boosts the accuracy of state-of-the-art non-pre-training independent-embedding models respectively by 9.1%, 4.2%, and 6.6% in terms of R@sum score on Flickr30K, MS-COCO 1K and MS-COCO 5K datasets. The retrieval accuracy also outperforms most existing interactive-embedding models with orders of magnitude decrease in retrieval time. The source code is available at: https://github.com/Zjamie813/SelfAlign.

28 October 2024
Verified
Conference Paper
Open access
XQXiangyan Qu
Xiangyan Qu
JYJing Yu
Jing Yu
Keke GaiKeke Gai
Keke Gai
JZJiamin Zhuang
Jiamin Zhuang
YTYuanmin Tang
Yuanmin Tang
QWQi Wu
Qi Wu
Recent work shows that documents from encyclopedias serve as helpful auxiliary information for zero-shot learning. Existing methods align the entire semantics of a document with corresponding images to transfer knowledge. However, they disregard that semantic information is not equivalent between them, resulting in a suboptimal alignment. In this work, we propose a novel network to extract multi-view semantic concepts from documents and images and align the matching rather than entire concepts. Specifically, we propose a semantic decomposition module to generate multi-view semantic embeddings from visual and textual sides, providing the basic concepts for partial alignment. To alleviate the issue of information redundancy among embeddings, we propose the local-to-semantic variance loss to capture distinct local details and multiple semantic diversity loss to enforce orthogonality among embeddings. Subsequently, two losses are introduced to partially align visual-semantic embedding pairs according to their semantic relevance at the view and word-to-patch levels. Consequently, we consistently outperform state-of-the-art methods under two document sources in three standard benchmarks for document-based zero-shot learning. Qualitatively, we show that our model learns the interpretable partial association. Code is available at https://github.com/MorningStarOvO/EmDepart.

3 December 2024
Verified
Conference Paper
Open access
ZZZhitong Zhu
Zhitong Zhu
JYJing Yu
Jing Yu
Keke GaiKeke Gai
Keke Gai
JZJiamin Zhuang
Jiamin Zhuang
GGGaopeng Gou
Gaopeng Gou
Gang XiongGang Xiong
Gang Xiong

27 October 2025
Verified
Conference Paper
Open access
YLYili Li
Yili Li
GXGang Xiong
Gang Xiong
GGGaopeng Gou
Gaopeng Gou
XQXiangyan Qu
Xiangyan Qu
JZJiamin Zhuang
Jiamin Zhuang
JSJunzheng Shi
Junzheng Shi