In recent years, the field of unmanned aerial vehicle (UAV) tracking has grown rapidly, finding numerous applications across various industries. While DCF-based trackers remain the most efficient and widely used in UAV tracking, recently lightweight CNN-based trackers using filter pruning have also demonstrated impressive efficiency and precision. However, the performance of these lightweight CNN-based trackers are still far from satisfactory. In the of generic visual tracking, emerging ViT-based trackers have shown great success by using cross-attention instead of correlation operation, enabling more effective capturing of relationships between the target and the search image. But to our knowledge, the UAV tracking community has not yet well explored the potential of vision transformers for more effective and efficient template-search coupling for UAV tracking. In this paper, we propose an efficient ViT-based tracking framework for real-time UAV tracking. Our framework integrates feature learning and template-search coupling into an efficient one-stream ViT to avoid an extra heavy relation modeling module. However, we observe that it tends to weaken the target information through transformer blocks due to the significantly more background tokens. To address this problem, we propose to maximize the mutual information between the template image and its feature representation produced by the ViT. The proposed method is dubbed TATrack. Additionally, to further enhance efficiency, we introduce a novel mutual information maximization-based knowledge distillation, which strikes a better trade-off between accuracy and efficiency. Exhaustive experiments on five benchmarks show that the proposed tracker achieves state-of-the-art performance in UAV tracking. Code is released at: https://github.com/xyyang317/TATrack.
12 January 2026 in IEEE Transactions on Circuits and Systems for Video Technology
In Unmanned Aerial Vehicle (UAV) tracking, discriminative correlation filters (DCF) are popular for their speed, making them ideal for real-time use with limited resources. Recently, lightweight convolutional neural networks (CNNs) have offered a new approach. Through filter pruning, these CNNs maintain high accuracy and efficiency, making them a strong alternative, especially for greater precision. Despite these advancements, the potential of pure vision transformers (ViTs) in UAV tracking remains largely untapped, especially based on the paradigm of conditional computation. In this work, we introduce an adaptive and background-aware Vision Transformer (Aba-ViT) and leverage it to develop a real-time UAV tracking framework called Aba-ViTrack. The proposed Aba-ViT exploits an adaptive and background-aware token computation method to reduce inference time. This approach adaptively discards tokens based on learned halting probabilities, which a priori are higher for background tokens than target ones. To further improve efficiency, this paper proposes a novel classroom-style learning (CSL) approach, where robustness knowledge is transmitted vertically from teacher to students, and generalization capability is enhanced through horizontal mutual learning among students. This method is used to compress Aba-ViTrack, resulting in Aba-ViTrack++. The upgraded version achieves a better balance between accuracy and efficiency in real-time UAV tracking. This version achieves a better balance between accuracy and efficiency for real-time UAV tracking. Extensive experiments on six UAV tracking benchmarks demonstrate that the proposed method achieves state-of-the-art performance in UAV tracking. The code is available at https://github.com/xyyang317/Aba-ViTrack.
Achieving fast and robust object tracking is critical for real-time Unmanned Aerial Vehicle (UAV) applications, where targets often move unpredictably and environmental conditions can rapidly change. In this paper, we propose the Search-Region Adaptive Mamba-ViT Tracker (SAMViTrack), a novel framework that combines the efficiency of Mamba attention with the powerful feature extraction capabilities of Vision Transformer (ViT). Our tracker dynamically adjusts the search region based on the target’s motion and environmental context, ensuring precise tracking even under challenging conditions such as occlusions, fast motion, and scale variations. By integrating an adaptive search mechanism, our SAMViTrack significantly reduces computational overhead without compromising accuracy, making it suitable for real-time deployment on UAVs with limited onboard resources. Extensive experiments on benchmark datasets demonstrate that our method outperforms both traditional and modern trackers, achieving superior accuracy and robustness with improved efficiency. The proposed tracker sets a new baseline, especially by combining Mamba and ViT, for UAV tracking by offering a balance between speed, accuracy, and adaptability in dynamic environments.
Efficient UAV tracking faces challenges due to limited onboard resources, short battery life, and payload constraints. discriminative correlation filters (DCF)-based trackers have traditionally been preferred for their efficiency. However, recent lightweight deep learning (DL) trackers, leveraging model compression, exhibit significant CPU efficiency and precision. Unfortunately, existing model compression approaches have struggled with maintaining tracking precision, especially at higher compression rates. This paper addresses this limitation with a novel approach, called disentangled representation learning with mutual information maximization (DR-MIM). DR-MIM has the potential to enhance the accuracy and efficiency of DL-based trackers, particularly in UAV tracking. The primary concept underpinning this approach is disentangled representation which bifurcates the feature space into two distinct categories: identity-related and identity-unrelated elements. This bifurcation serves as a critical component, powering the heightened level of precision and efficiency attained by DL-based trackers. The focus of our work is predominantly on the employment of exclusively identity-related features. As a result of this focus, there’s a noticeable escalation in the efficacy of the feature representation. To further enhance efficiency without compromising accuracy, we have quantified the network, which has been named DR-MIM (v2). Exhaustive experiments conducted across four aerial tracking benchmarks have demonstrated the superiority of the proposed method. A standout performance on VisDrone2018 showcases DR-MIM (v2)’s impressive GPU speed of 658 FPS, coupled with a maintained precision of 82.1%, underscoring the potential of our approach in real-world UAV tracking scenarios.