Large-scale image and text representation learning is critical in determining the performance of multimodal tasks involving images and text, such as visual question answering and image captioning. Most existing research on large-scale image and text representation learning relies on Transformer networks for pre-training, i.e., learning generic semantic representations from large-scale image-to-text pairs. These representations are then fine-tuned and transferred to downstream multimodal tasks. This paper first provides a brief analysis of the advantages of pre-training models. It then comprehensively summarizes the relevant research on large-scale image and text representation learning based on pre-training. The focus is on pre-training model architectures, pre-training tasks, and image-text datasets. Finally, we provide a summary and outlook of large-scale image and text representation learning.

This paper introduces ProFiC-VLP, a fine-grained vision-language pre-training (VLP) model featuring progressive freezing and feedback-controlled cropping. Our ProFiC-VLP model addresses the challenge of fine-grained image-text semantic relationship alignment in VLP, specifically in aligning visual regions with their corresponding textual phrases. ProFiC-VLP integrates three levels of image-text semantic alignment: global alignment, object alignment, and relationship alignment. It employs vision transformer for image encoding and a semantic parser to decompose text into multi-level semantic structures, which are then encoded into text semantics using lightweight querying transformers. To prevent overfitting during pre-training, ProFiC-VLP introduces a progressive freezing strategy that aligns multi-level textual representations with the corresponding image concepts. Furthermore, a novel language generation approach with a feedback-controlled cropping loss function is proposed for relationship alignment pre-training, which effectively localizes image regions corresponding to specific textual phrases. Experimental results show that ProFiC-VLP outperforms existing models across various vision-language tasks, especially in cross-modal reasoning scenarios such as visual grounding and image caption generation. Visualization experiments further underscore ProFiC-VLP’s ability to achieve fine-grained alignment of semantic relationships, significantly enhancing its capacity to address complex vision-language challenges.

Symbolic music understanding is a critical challenge in artificial intelligence. While traditional symbolic music representations like MIDI capture essential musical elements, they often lack the nuanced expression in music scores. Leveraging the advancements in multimodal pre-training, particularly in visual-language pre-training, we propose a groundbreaking approach: the Score Images as a Modality (SIM) model. This model integrates music score images alongside MIDI data for enhanced symbolic music understanding. We also introduce novel pre-training tasks, including masked bar-attribute modeling and score-MIDI matching. These tasks enable the SIM model to capture music structures and align visual and symbolic representations effectively. Additionally, we present a meticulously curated dataset of matched score images and MIDI representations optimized for training the SIM model. Through experimental validation, we demonstrate the efficacy of our approach in advancing symbolic music understanding.

Vision–language pre-training (VLP) faces challenges in aligning hierarchical textual semantics (words/phrases/sentences) with multi-scale visual features (objects/relations/global context). We propose a hierarchical VLP model (HieVLP) that addresses such challenges through semantic decomposition and progressive alignment. Textually, a semantic parser deconstructs captions into word-, phrase-, and sentence-level components, which are encoded via hierarchical BERT layers. Visually, a Swin Transformer extracts object- (local), relation- (mid-scale), and global-level features through shifted window hierarchies. During pre-training, a freezing strategy sequentially activates text layers (sentence→phrase→word), aligning each with the corresponding visual scales via contrastive and language modeling losses. The experimental evaluations demonstrate that HieVLP outperforms hierarchical baselines across various tasks, with the performance improvements ranging from approximately 3.2% to 11.2%. In the image captioning task, HieVLP exhibits an average CIDEr improvement of around 7.2% and a 2.1% improvement in the SPICE metric. For image–text retrieval, it achieves recall increases of 4.7–6.8%. In reasoning tasks, HieVLP boosts accuracy by 2.96–5.8%. These results validate that explicit multi-level alignment enables contextually coherent caption generation and precise cross-modal reasoning.

Atmospheric turbulence refers to the random and irregular airflow in the three-dimensional Earth’s atmosphere, and the atmospheric refractive index structure constant $C_n^2$ is often used to quantitatively express the intensity of atmospheric turbulence. In this study, we first analyze the atmospheric wind field profiles and $C_n^2$ profiles measured by the Mesosphere–Stratosphere–Troposphere (MST) radar station in Qinzhou, Guangxi, which was established under the National Meridian Project. The results are compared with the calculation results from the NCEP dataset, confirming the stability and reliability of the data from the Qinzhou, Guangxi MST radar station. Furthermore, we statistically analyzed its diurnal variation characteristics on 4 October 2024, and monthly seasonal distribution characteristics from August 2024 to July 2025. The results show that the diurnal variation of the atmospheric refractive index structure constant is stronger during the day than at night, and stronger in spring and autumn than in summer and winter, which is consistent with the expected results. Finally, we separately analyzed the capture of the intensely turbulent typhoon “Capricorn” on 7 September 2024, by the MST radar-measured wind field, atmospheric refractive index structure constant, and surface meteorological stations. Linear fitting was performed on the horizontal wind vertical shear perturbations and atmospheric refractive index structure constant perturbations caused by the turbulence event, with a coefficient of determination $R^2=0.70128$. This result suggests that, during severe typhoon events, vertical wind shear serves as the primary driving mechanism for the enhancement of atmospheric optical turbulence.