1 December 2023 in IEEE Transactions on Instrumentation and Measurement
Thermal infrared (TIR) target tracking task is not affected by illumination changes and can be tracked at night, on rainy days, foggy days, and other extreme weather; so it is widely used in auxiliary driving, unmanned aerial vehicle reconnaissance, video surveillance, and other scenes. However, the TIR target tracking task also presents some challenges, such as intensity change, occlusion, deformation, similarity interference, and so on. These challenges significantly affect the performance of the TIR target tracking methods. To resolve these challenges in the TIR target tracking scenarios, numerous tracking methods have appeared in recent years. The purpose of this article is to give a comprehensive review and summary of the research status of the TIR target tracking methods. We first classify the TIR target tracking methods according to their frameworks and briefly summarize the advantages and disadvantages of different tracking methods, which can better understand the current research progress of the TIR target tracking methods. Next, the public datasets/benchmarks for performance testing of the TIR target tracking methods are introduced. Subsequently, we demonstrate the tracking results of several representative tracking methods on some datasets/benchmarks to more intuitively show the progress of the TIR target tracking methods made in the current research. Finally, we discussed the future research direction in an attempt to promote the better development of the TIR target-tracking tasks.
31 July 2024 in IEEE Transactions on Instrumentation and Measurement
RGBT tracking seeks to leverage both visible and thermal infrared images to enhance the robustness of target tracking. This method makes up for the limitations of single-sensor tracking. The RGB and thermal infrared images complement each other effectively, enabling the tracker to operate seamlessly in complex environments day and night. However, RGBT tracking still faces challenges such as thermal crossover, occlusion, illumination changes, rapid target movement, and background clutter. These problems hinder the performance improvement of RGBT trackers. Researchers have proposed numerous RGBT tracking methods in response to these challenges over time. This paper aims to comprehensively review and summarize the latest research progress in RGBT tracking. First, we categorize the existing tracing methods according to the RGBT tracing framework. Then, we introduce the datasets/benchmarks used to evaluate RGBT tracking methods. Subsequently, we present the results of representative tracking methods on multiple testing benchmarks, providing a more intuitive presentation of current research advances in the field of RGBT tracking. Finally, research directions are discussed to further advance the development of RGBT tracking.
5 June 2025 in IEEE Sensors Journal
RGBT tracking (RGB-Thermal) usually uses an RGB tracker as the base model, and then uses the RGBT dataset to fully fine-tune the model. These methods ignore the differences in target features in the RGB domain and the TIR domain. At the same time, existing RGBT trackers match the spatial features of the initial template and the search image, ignoring the role of temporal information in RGBT tracking, resulting in the failure of the tracker to track in complex scenarios such as change in the appearance of the target and occlusion. To address the above problems, we propose a simple and efficient tracker called STTrack. The tracker adopts a symmetric dual-stream structure, which consists of several FT Transformer(Fine Tuning Transformer) Encoders, a prediction head, and an online update module. Specifically, the FT Transformer Encoder first adds some trainable parameters to the frozen pre-trained RGB-based tracker, transfers the feature extraction capability from the RGB domain to the TIR domain, and enhances the model’s perception of cross-modal data; secondly, the output features of the RGB and TIR modalities are fused and fed into the prediction head to obtain the target’s position; finally, the online update module obtains an online template with temporal information, which complements the spatial information provided by the initial template. The spatio-temporal information provided by the dual templates improves the RGBT tracker’s ability to locate targets in complex environments. Extensive quantitative and qualitative experiments demonstrate that our approach achieves state-of-the-art performance on four most popular RGBT benchmarks and runs at 32FPS in real time.
2 June 2026 in IEEE Sensors Journal
Robust multi-modal sensing is essential for reliable object tracking in complex environments. Although RGB-Thermal (RGBT) sensing leverages complementary spectral modalities to enhance tracking performance, existing methods often rely on static template matching, thereby neglecting the inherent temporal continuity of sensor streams. This limitation compromises tracking stability under challenging sensing conditions, such as abrupt illumination changes, severe occlusions, and thermal crossover. To address these challenges, this paper proposes TFTrack, A new type of closed-loop video stream RGBT tracking framework has adopted a deep-collaborative two-component time design: a dynamic template library, which provides spatial priors for trajectory modeling by storing the prediction box results of historical templates; and autoregressive time label propagation, which compensates for the continuity deficiency of pure template methods. Unlike traditional sparse sampling paradigms, TFTrack exploits continuous temporal informatics by integrating historical appearance features and motion trajectories within a dynamic template library, ensuring consistent perception across time. Furthermore, to maximize multi-sensor synergy, we develop an Input Feature Guide Attention Fusion (IFGAFusion) module. By dynamically recalibrating spatial importance weights under the guidance of input features, this module effectively fuses heterogeneous data streams from RGB and Thermal Infrared (TIR) sensors. Extensive qualitative and quantitative experiments demonstrate that the proposed method achieves state-of-the-art performance. Notably, TFTrack operates at 30 FPS on the LasHeR benchmark with a precision rate of 74.7% and a success rate of 59.6%, satisfying the stringent real-time and reliability requirements for practical vision-based sensing systems.