Remote sensing object detection (RSOD) is crucial for various monitoring and measurement applications due to its non-contact, large-scale and real-time observation, such as autonomous driving, scene awareness, and environmental surveillance. Despite significant advances driven by deep neural networks, RSOD faces challenges such as directional ambiguity, scale variations, and densely packed tiny objects, limiting its effectiveness in diverse scenarios. To address these challenges, we explore a novel end-to-end deep RSOD method, termed Gradual Semantic Interaction Network (GSINet). Specifically, we first design a Gradual Semantic Information Interaction Structure (GSIIS), which integrates contextual information across network layers from shallow to deep, enhancing semantic comprehension and detection precision. Second, we propose a novel Dual Attention Mechanism (DAM) that integrates our proposed Adaptive Context Feature Fusion Module (ACFFM) with Bi-Level Routing Attention (BRA) to optimize feature representation, ACFFM enhances context-aware feature fusion by capturing both local and global information, whereas BRA routes attention across different scales, together, they enhance tiny object detection in remote sensing images by reducing background interference and improving feature discrimination. Finally, we utilize the normalized Wasserstein distance (NWD) in place of the traditional Intersection over Union (IoU) metric to better manage object scale variations and location deviations. The extensive experimental evaluation on three public datasets demonstrates that the proposed method exhibits superior performance compared to state-of-the-art detectors, especially in detecting tiny objects. In particular, GSINet achieves a mean average precision (mAP) of 86.7% on the DIOR dataset, 95.01% on the NWPU VHR-10 dataset, and 96.7% on the RSOD dataset