Generating accurate and contextually coherent descriptions in remote sensing image captioning (RSIC) remains a formidable challenge, owing to the complex spatial structures of aerial and satellite imagery, as well as the requirement for high-level semantic interpretation. Conventional RSIC models predominantly rely on visual features alone, thereby overlooking rich linguistic priors that could significantly improve narrative consistency and scene understanding. To address this, we propose DPPF: Dual-Path Pre-Fusion with Semantic-Guided Encoding, a novel framework that integrates visual and semantic modalities at an early stage to leverage their complementary strengths. Specifically, we first employ the CLIP vision encoder to extract fine-grained patch-level visual features, which are fed as key-value pairs into the cross-attention layers of a transformer decoder, offering precise spatial grounding for word-level generation. Parallelly, we generate a scene-level caption using a pre-trained large language model (LLM) and encode it into a global semantic vector via the CLIP text encoder. Crucially, we further introduce a pre-fusion mechanism that concatenates this semantic vector with word embeddings to construct the input sequence for the decoder's masked self-attention, thereby allowing the language model to generate captions conditioned on holistic scene semantics from the outset. By separating visual and semantic encoding while injecting semantic context early, our method achieves balanced and effective multimodal interaction. Comprehensive experiments on three benchmark RSIC datasets show that DPPF achieves competitive performance compared to current state-of-the-art approaches, demonstrating the advantages of semantic-guided pre-fusion for remote sensing image captioning.
1 January 2026 in IEEE Geoscience and Remote Sensing Letters
Human action understanding aims to detect action instances and classify them in videos. Existing methods capture contextual dependencies through temporal feature aggregation or attention-based interactions. However, these methods primarily rely on visual features to establish relationships, making them susceptible to interference from background content or irrelevant action frames. To address this limitation, we propose Hybrid Causal Action Reasoning (HCAR), a framework for modeling causal correlations between action proposals. Specifically, we design a Hybrid Causality-Aware Attention (HCAA) to refine the semantic representations of proposal classes. To further characterize relations between adjacent proposals, an ordered relation branch evaluates the refined representations in temporal order, while a hyperbolic branch measures hierarchical semantic relatedness between action categories in the Poincaré ball. The outputs of these branches are combined to obtain causal correlation scores, which are subsequently used to refine action proposals. Extensive experiments on THUMOS14 and ActivityNet-1.3 demonstrate the effectiveness of our HCAR.