Hyperspectral videos contain richer spectral and physical features than RGB videos and thus have greater potential for use in object tracking. The mainstream hyperspectral object tracking approach involves the integration of multiple RGB-based video tracking models. Although ensembles of multiple models can effectively utilize spectral information and improve tracker performance, this approach has high computational complexity, making it difficult to meet the real-time requirements of video object tracking. To bridge the gap, we propose a new hyperspectral object tracking framework (HotMoE) based on Mixture-of-Experts (MoE). HotMoE leverages a divide-and-conquer strategy, where only a subset of expert models is computed for each input, reducing computational complexity while maintaining performance. In this paper, we first design a splitter to group multiple spectral bands into multiple false-color images based on spectral correlations. Then, we design a hyperspectral MoE router that can adaptively learn to aggregate spectral image feature information and route it to suitable experts. Different experts can handle various scenarios, and HotMoE effectively utilizes the capabilities of different experts to obtain better overall performance. Compared with previous state-of-the-art hyperspectral object tracking networks, our model has significantly reduced inference time and performs well, with a processing speed of 43.7 FPS and an AUC of 0.704 with the HOT2022 dataset. The codes and models are available at https://github.com/supertyd/hotmoe.

Many spatiotemporal image fusion methods in remote sensing have been developed to blend highly resolved spatial images and highly resolved temporal images to solve the problem of a trade-off between the spatial and temporal resolution from a single sensor. Yet, none of the spatiotemporal fusion methods considers how the various temporal changes between different pixels affect the performance of the fusion results; to develop an improved fusion method, these temporal changes need to be integrated into one framework. Adaptive-SFSDAF extends the existing fusion method that incorporates sub-pixel class fraction change information in Flexible Spatiotemporal DAta Fusion (SFSDAF) by modifying spectral unmixing to select spectral unmixing adaptively in order to greatly improve the efficiency of the algorithm. Accordingly, the main contributions of the proposed adaptive-SFSDAF method are twofold. One is to address the detection of outliers of temporal change in the image during the period between the origin and prediction dates, as these pixels are the most difficult to estimate and affect the performance of the spatiotemporal fusion methods. The other primary contribution is to establish an adaptive unmixing strategy according to the guided mask map, thus effectively eliminating a great number of insignificant unmixed pixels. The proposed method is compared with the state-of-the-art Flexible Spatiotemporal DAta Fusion (FSDAF), SFSDAF, FIT-FC, and Unmixing-Based Data Fusion (UBDF) methods, and the fusion accuracy is evaluated both quantitatively and visually. The experimental results show that adaptive-SFSDAF achieves outstanding performance in balancing computational efficiency and the accuracy of the fusion results.

Compared to conventional RGB images, hyperspectral images offer a more comprehensive range of spectral information, encompassing both visible and infrared bands. This enhanced spectral information facilitates trackers in effectively differentiating the target object from background clutter, realizing more robust recognition. However, hyperspectral video cameras exhibit variability in terms of spectral ranges and band numbers, producing distinct modalities. Developing specialized networks for each modality proves to be inefficient and time-consuming, while networks trained for one modality struggle to perform well on others, generating unsatisfactory outcomes. To confront these challenges encountered in various tracking scenarios, HyperTrack is proposed as a unified object tracking network tailored for hyperspectral videos. The proposed HyperTrack can be employed individually for single object tracking across three different modalities of hyperspectral videos: near infrared (NIR), visible (VIS), and red-to-near infrared (RedNIR). Specifically, since hyperspectral data have multiple bands, a band gate module is introduced into the network to enable it to select bands from hyperspectral images, thereby reducing the dimensionality for hyperspectral images. Furthermore, in order to effectively utilize the variability amongst the three different modal data, a computationally simple band embedding module is introduced to improve the object tracking performance of different modal hyperspectral videos. Additionally, a hybrid attention module is devised to efficiently extract and interact features between the template and search at each stage. As a unified network, HyperTrack achieves state-of-the-art comprehensive results across three different types of hyperspectral videos, particularly excelling with VIS and NIR type data. The code and models are publicly available at https://github.com/supertyd/HyperTrack.

The real-time performance and precision requirements posed by the ranging method for proximity fuze determines the effect and hit accuracy of the explosive device on which it is mounted. This further demands not only high spectrum resolution near the burst point for the fuze, but also the ability to monitor the spectrum of the entire observation area. This paper proposes a ranging method for proximity fuze based on warped fast Fourier transform (WFFT). By selecting appropriate all-pass warping factors, our proposed method can improve the spectrum precision within a specific spectrum region observed without increasing signal processing points. By fitting non-uniform filter banks, the diffused energy of target object in high-frequency region is accumulated, which can be regarded as improving the signal-to-noise ratio (SNR) in high-frequency region, high-frequency information can be accurately captured, thereby facilitating real-time ranging of proximity fuze with higher precision. The experimental results have verified the feasibility and effectiveness of our proposed method in practical proximity fuze ranging.

A multispectral filter array (MSFA) is a typical snapshot spectral imaging technique, but a single sensor is limited by its narrow overall spectral range and limited number of bands. To obtain a wider spectral range and more imaging bands, the use of multiple MSFA sensors has become an effective solution. However, the existing demosaicing algorithms were designed mainly for single-MSFA systems and lack effective methods for jointly reconstructing MSFA images that are synchronously acquired from multiple sources. To address this issue, we propose a novel multi-MSFA demosaicing network, the KAN-enhanced siamese transformer pyramid network (STP-KAN). This network is built on a siamese feature pyramid architecture and uses a Kolmogorov–Arnold Network (KAN)-based half-shuffle multi-head self-attention as its basic module, effectively leveraging the long-range modeling capabilities of transformers and the powerful nonlinear representation ability of the KAN. A cross-attention mechanism is utilized to efficiently share features among multi-MSFA images. A spatial-spectral attention mechanism is embedded into the skip connections of the network to fuse deep spatial and spectral features. MSFA demosaicing experiments conducted on the ARAD-1K open-source natural scene dataset and our self-built AVIRIS-NG remote sensing dataset demonstrate the superior performance of our proposed method. The source code is available at https://github.com/Stephen-future/STP-KAN.