Cross-modal retrieval aims to elucidate information fusion, imitate human learning, and advance the field. Although previous reviews have primarily focused on binary and real-value coding methods, there is a scarcity of techniques grounded in deep representation learning. In this paper, we concentrated on harmonizing cross-modal representation learning and the full-cycle modeling of high-level semantic associations between vision and language, diverging from traditional statistical methods. We systematically categorized and summarized the challenges and open issues in implementing current technologies and investigated the pipeline of cross-modal retrieval, including pre-processing, feature engineering, pre-training tasks, encoding, cross-modal interaction, decoding, model optimization, and a unified architecture. Furthermore, we propose benchmark datasets and evaluation metrics to assist researchers in keeping pace with cross-modal retrieval advancements. By incorporating recent innovative works, we offer a perspective on potential advancements in cross-modal retrieval.

Cross-modal retrieval is crucial for achieving accurate and efficient information retrieval by establishing semantic correlations between heterogeneous images and text. However, traditional image-text training sets suffer from information asymmetry, which includes short lengths and limited sentence structures. This phenomenon often results in insufficient representations of essential visual information. We introduce Rich-Dataset, which offers extensive semantic information. It includes diverse real-life image-text pairs and AI-generated content across domains such as news, entertainment, education, and posters. Compared with classic benchmarks such as Flickr30k and MS-COCO, RichDataset exhibits a novel and balanced distribution. Existing cross-modal retrieval models face challenges in extracting distinct features from the emerging data, leading to low retrieval accuracy. We propose SGG-MVAR, a comprehensive retrieval model guided by multiview scene information and semantic relationships. Leveraging a scene knowledge database, our model parses scene graphs and identifies differences in attributes and relationships. We conduct extensive experiments to evaluate our proposed dataset and model. All experimental results consistently demonstrate a significant improvement in recall for cross-modal retrieval.