23 September 2024 in ACM Transactions on Multimedia Computing, Communications, and Applications
Cross-modal hashing is widely used for efficient similarity searches, improving data processing efficiency, and reducing storage costs. Existing cross-modal hashing methods primarily focus on centralized training scenarios, where fixed-scale and fixed-category multi-modal data is collected beforehand. However, these methods often face challenges associated with the potential risk of privacy breaches and high data communication costs during data transmission in real-world multimedia retrieval tasks. To tackle these challenges, in this article, we propose an efficient privacy-enhanced prototype-based federated cross-modal hashing (PEPFCH). In PEPFCH, we integrate local and global prototypes in order to effectively capture the distinctive traits of individual clients, while also harnessing the collective intelligence of the entire federated learning system. Moreover, to ensure the security of prototype information and prevent its disclosure during the aggregation process, we use a prototype encryption transmission mechanism to encrypt the prototype information before transmission, making it challenging for attackers to gain access to sensitive data. Additionally, to facilitate personalized federated learning and alleviate the issue of parametric catastrophic forgetting, we establish the image and text hyper-networks for each client and adopt a hyper-network extension strategy to selectively preserve and update previously acquired knowledge when acquiring new concepts or categories. Comprehensive experiments highlight the efficiency and superiority of our proposed method. To enhance research and accessibility, we have publicly released our source codes at: https://github.com/vindahi/PEPFCH .
19 October 2026 in ACM Transactions on Multimedia Computing, Communications, and Applications
Multi-modal hashing converts high-dimensional features into compact binary codes while preserving semantic similarity, thereby achieving high performance with lower storage and computational costs. However, existing methods generally overly depend on discrete supervisory information, ignoring the ambiguity of labels and intra-class diversity. To address these challenges, we propose a novel
S
emantic
S
elf-refinement
M
ulti-modal
H
ashing
(SSMH), which adaptively learns stable semantic representations through a unified end-to-end semantic learning. First, we dynamically integrate inter- and intra- modal correlations to achieve semantically consistent multi-modal integrated representation. Then, we generate prototype-guided binary codes to explicitly align multi-modal representations with label semantics, thereby strengthening supervision and mitigating the effect of ambiguous labels. Extensive experiments show SSMH outperforms state-of-the-art methods in multi-modal retrieval. The source code can be found at https://github.com/vindahi/SSMH.
1 January 2026 in IEEE Transactions on Multimedia
1 January 2026 in IEEE Transactions on Multimedia