Remote sensing image object detection presents significant challenges, due to the difficulty in accurately predicting the rotational angles of ground-oriented objects, coupled with issues such as the false and missed detection caused by insufficient object information. Moreover, traditional convolutional neural networks are inherently limited in their capacity to capture global contextual information. To address these challenges, a DETR-based remote sensing image object detection model is designed for oriented objects. Except for the backbone, transformer encoders and decoders, scenario query guiding modules, oriented boxes refinement modules, auxiliary multiple detectors, and oriented boxes denoising modules are also designed and included in our network. The scenario query guiding module is proposed that implicitly guides the decoder to focus more on object classification information specific to that scene when inferring. The multiple deformable attention mechanism is improved to the oriented one and utilized into the oriented boxes refinement module which repeatedly corrects the oriented boxes, enhancing the network’s ability to predict the oriented boxes precisely. At the same time, the improved auxiliary multiple detectors and oriented boxes denoising module are applied only for the training process to enhance the learning ability of the encoder and decoder for oriented objects. The ablation experiments proved the effectiveness of the designed modules. The detection accuracy of our network on DOTAv1.0 (76.77%) and HRCS2016 (97.01%) is improved compared with the state-of-the-art methods, which are especially significantly higher than DETR detection algorithms.
RGB and thermal visual tracking is a highly effective solution for complex environments. However, existing multimodal trackers still face significant limitations. They often apply uniform fusion weights across the entire image. They also ignore the distinct physical properties of different modalities and lack historical temporal modeling. To address these issues, we propose a novel video-level framework named Mutual Enhancement Fusion Tracker (MEFT). First, we introduce a modality-aware mutual enhancement module. This module explicitly transfers visual texture from the RGB modality to the thermal modality. Simultaneously, it transfers heat saliency from the thermal modality to the RGB modality. Second, we design an adaptive spatial gating network. This network generates normalized pixel-level weights to ensure an optimal combination of features at different locations. Finally, the fused representations are processed by a spatiotemporal sequence backbone. Extensive experiments on four benchmark datasets demonstrate that our method achieves competitive performance with state-of-the-art RGB-T trackers. For instance, our tracker achieves 60.3% SR on LasHeR and 67.5% MSR on RGBT234. MEFT significantly improves both tracking accuracy and overall reliability in a wide range of challenging real-world scenarios.
Deep learning has significantly improved the quality of infrared and visible image fusion. However, existing mainstream deep fusion networks often come with complex architectures and a large number of parameters. While general lightweight techniques can effectively reduce model complexity, they often weaken feature interactions during the lightweighting process, resulting in the loss of complementary texture and thermal information in fused images and making it difficult to balance fusion performance and model efficiency. To address these issues, this paper constructs an infrared and visible image fusion network based on a self-compensating lightweight convolution mechanism, named LWC-DenseFuse. The core of the network lies in a self-compensating lightweight convolution module, which goes beyond conventional convolution replacement and explicitly addresses feature degradation introduced by lightweight design. The module decouples spatial and channel correlations of standard convolution through depthwise convolution and pointwise convolution, while incorporating a channel attention mechanism to adaptively enhance salient features. Additionally, channel shuffle technology is employed to promote information exchange between groups, thereby enhancing feature interaction and compensating for the loss of critical information caused by lightweight design. To further improve the representation capability of the lightweight network during optimization, a staged training strategy with progressive loss weighting is introduced. Experimental evaluations demonstrate that the proposed fusion network significantly reduces the number of model parameters while ensuring real-time inference performance. Meanwhile, it effectively alleviates the performance degradation typically associated with lightweight architectures, as evidenced by improvements in information entropy and visual fidelity.