6 March 2023 in IEEE Transactions on Geoscience and Remote Sensing
Visual object tracking is a fundamental task in computer vision, and thrived in recent decades. With the development of snapshot hyperspectral sensors, efforts have been made to exploit tracking the object with hyperspectral (HS) videos to overcome the inherent limitation of RGB images. Existing HS tracking algorithms extract the deep features from image data separately, which break the interaction information between bands. Therefore, the discrimination ability of HS trackers is limited and the efficiency of the existing HS algorithms is low. In this paper, a novel algorithm (CBFF-Net) is proposed for HS object tracking to improve the discrimination ability and reduce the computational complexity. Specifically, the backbone and head network are implemented with modules of a transferred RGB object tracking network to carry out the HS target tracking task while maintaining the discrimination ability learned from RGB data. Moreover, a bi-directional multiple deep feature fusion (BMDFF) module is proposed to fuse the features extracted from different bands of the HS images, and a cross-band group attention (CBGA) module is introduced to learn interaction information across bands of the HS images. Experiments results indicate the superiority in performance of CBFF-Net, and it runs at 24 frames per second.
17 October 2023 in IEEE Transactions on Geoscience and Remote Sensing
With the abundant spectral information, the hyperspectral images could be benefit for tracking the target in varies application scenarios. Most of the predominant hyperspectral object tracking methods were based on transferred RGB object tracking networks, since the lack of training data. Different strategies of processing the hyperspectral images to adapt the transferred RGB object tracking networks have been exploited. However, the existing strategies led to lose the spectral information or the interaction information between bands, and shown limited performances. In this paper, a novel transformer-based hyperspectral object tracking algorithm (Trans-HST) is proposed to make advantages of the spectral information with transformer modules. In Trans-HST, the cross band-groups of features enhancement (CBFE) is introduced to reduce the negative effects of the interaction information loss. To address the problem of spectral information loss, the transformer-based deep features fusion (TDFF) fuses the deep features corresponding to different group of bands in the hyperspectral images, and integrates the deep feature corresponding to the original hyperspectral images into the fused features. Experiments on the commonly used hyperspectral object tracking dataset have been applied to verify the effectiveness of the two proposed modules, and indicate the superior performance of the Trans-HST comparing with other RGB and hyperspectral trackers.