Aerial object tracking has recently shown great potential in the field of remote sensing. However, small objects with limited feature information pose a huge challenge to aerial trackers. Despite significant improvements, most trackers still struggle to capture enough discriminative features and to overcome background disturbances. In this work, we propose an efficient aerial tracker (SmallTrack) based on the Siamese network to improve the discrimination of small objects. It consists of two effective modules, namely Wavelet Pooling Layer (WPL) and Graph Enhanced Module (GEM). First, WPL decomposes the input into four subbands via wavelet domain learning, and fully utilizes the high- and low-frequency information in the subbands to preserve the discriminative features of small objects. Second, GEM embeds the pixels on the classification responses as nodes in graph learning through graph neural networks, which naturally mines the similarity between pixels. Based on the pixel-level modulation constructed from graph theory, GEM enhances the understanding of small objects and highlights them in the classification responses. The proposed tracker achieves leading performance on five aerial benchmarks, while maintaining a high running speed of 72.5 frames/s. Besides, real-world tests on an aerial platform have proven the effectiveness of SmallTrack. The code and models are available at https://github.com/xyl-507/SmallTrack.
7 June 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Aerial tracking has received growing attention due to its broad practical applications. However, single-view aerial trackers are still limited by challenges such as severe appearance variations and occlusions. Existing multi-view trackers utilize cross-drone information to address these issues but struggle to overcome heterogenous differences. In this paper, we propose a novel Transformer-based consistent representation mining (CRM) module to capture invariant target information and suppress the heterogenous differences in cross-drone information. First, CRM divides the heterogenous input into regions and measures semantic relevance by modeling the relations between these regions. Then reliable target regions are roughly localized by selecting the top k most relevant regions. Next, the global perception is performed on these reliable regions via multi-head sparse self-attention, further enhancing the understanding of the target and suppressing background regions. In particular, CRM, as a plug-and-play module, can be flexibly embedded into different tracking frameworks (CRM-Siam and CRM-DiMP). Besides, the multi-view correction strategy is designed to ensure timely correction of multi-view information and full utilization of its own information. Extensive experiments on the multi-drone dataset, MDOT, demonstrate that CRM-assisted trackers effectively improve the accuracy and robustness of the multi-drone tracking system, outperforming other outstanding trackers. The code and models are available at https://github.com/xyl-507/CRM.
22 August 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Recently, RGB-T tracking has received increasing attention due to its robustness. However, existing RGB-T trackers mainly use cross-attention for modal feature interaction, limiting the utilization of complementary information. In addition, these trackers employ fixed dominant-auxiliary paradigms for feature fusion, ignoring modal quality fluctuations. To address these issues, we propose FMTrack, an effective framework for fully capturing complementary information. FMTrack consists of two key components, a frequency-aware interaction network (FIN) and a multi-expert fusion module (MEFM). To emphasize the valuable information in each modality, FIN utilizes frequency masks to perform high-pass and low-pass filtering on RGB and TIR data. FIN explicitly establishes cross-modal interactions via frequency domain learning, which facilitates the sharing of complementary information. Besides, MEFM extracts diverse features via the differentiated expert network and then adjusts feature combinations according to modal reliability, achieving deep understanding and flexible fusion of multimodal data. With FIN and MEFM, FMTrack makes full use of the advantageous information of each modality to highlight target representations, thus improving performance in complex scenes. Extensive experiments on four popular RGBT tracking datasets (LasHeR, VTUAV, RGBT234, and RGBT210) show that our FMTrack achieves leading performance. The code is available at https://github.com/xyl-507/FMTrack.
11 March 2025 in IEEE Transactions on Circuits and Systems for Video Technology
The introduction of natural language for vision-language (VL) tracking has been proven to improve performance. However, natural language remains under-explored in existing aerial trackers. Moreover, existing VL trackers ignore the misalignment of language with dynamic target states, which is prominent in complex UAV scenarios. In this work, we present AVLTrack, a flexible framework for aerial vision-language tracking. It consists of three key components, a dynamic sparse learning (DSL) module, an efficient Transformer backbone, and a multi-level language perception (MLP) strategy. First, DSL sparsely connects language and images via dynamic sparse attention, providing accurate multi-modal prompts. To adapt to target state variations, the sparsity in DSL is dynamically adjusted based on semantic information, flexibly highlighting target-specific tokens. Next, the Transformer backbone follows highly parallelized one-stream architectures, allowing efficient multi-modal feature extraction and interaction. Finally, MLP enables the iterative interaction of language and visual information, aiming to utilize language priori to guide the generation of discriminative visual features. Moreover, we construct the DTB70-NLP dataset to facilitate UAV vision-language tracking. Extensive experiments on WebUAV-3M and DTB70-NLP demonstrate the leading performance of AVLTrack compared to existing outstanding trackers while maintaining a high running speed of 80.5 FPS. The dataset and codes are available at https://github.com/xyl-507/AVLTrack.
9 August 2024 in IEEE Transactions on Instrumentation and Measurement
Aerial tracking has recently shown significant potential in vision-based measurement. Despite significant improvements, accurate occlusion tracking remains a difficult challenge. Existing training enhancement or global search struggles to handle occlusion in aerial views due to similar distractors and background interferences. To handle occlusion in aerial tracking, we propose a query-guided redetection tracker (QRDT) based on a Siamese neural network, improving the redetection discrimination from three stages. First, the query update branch is introduced to keep the target appearance dynamically updated via inter-frame information transfer. Next, we propose the cross fusion layer, which models the semantic correlation between the search feature and updated query feature, to draw attention to the occluded target. Finally, to address tracking failure and distraction from similar targets, the trajectory during full occlusion is reliably predicted by the Kalman filter. Our tracker achieves leading tracking performance on several benchmarks, with an average speed of 48.9 frames/s. The code and models are available at https://github.com/xyl-507/QRDT.
27 June 2025 in IEEE Transactions on Intelligent Transportation Systems
Object tracking is a basic task of the uncrewed aerial vehicle (UAV)-based intelligent visual perception system. The presence of similar targets and complex backgrounds in the airborne perspective poses significant challenges to aerial trackers. However, existing target-aware or distractor-aware trackers fail to capture discriminative cues from both target and background information in a balanced manner, resulting in limited improvement. To address these issues, this paper proposes a global agent-based Target-Distractor Aware Tracker (TDAT) to enhance the discrimination of the target. TDAT comprises two effective modules: a global agent generator and an interactor. First, the generator aggregates the target and background regions into representative agents and then performs self-attention on these agents to explicitly model the global relationships between the target and backgrounds. Next, the interactor realizes the bidirectional information interaction between global agents and local regions via self-attention. Based on the global dependencies encoded in global agents, the interactor extracts target-oriented features and enhances the understanding of the target. TDAT embedded with target-distractor awareness effectively widens the gap between target and background distractors. Experimental results on multiple UAV benchmarks show that TDAT achieves outstanding performance with a speed of 34.5 frames/s. The code is available at https://github.com/xyl-507/TDAT