Hyperspectral videos contain richer spectral and physical features than RGB videos and thus have greater potential for use in object tracking. The mainstream hyperspectral object tracking approach involves the integration of multiple RGB-based video tracking models. Although ensembles of multiple models can effectively utilize spectral information and improve tracker performance, this approach has high computational complexity, making it difficult to meet the real-time requirements of video object tracking. To bridge the gap, we propose a new hyperspectral object tracking framework (HotMoE) based on Mixture-of-Experts (MoE). HotMoE leverages a divide-and-conquer strategy, where only a subset of expert models is computed for each input, reducing computational complexity while maintaining performance. In this paper, we first design a splitter to group multiple spectral bands into multiple false-color images based on spectral correlations. Then, we design a hyperspectral MoE router that can adaptively learn to aggregate spectral image feature information and route it to suitable experts. Different experts can handle various scenarios, and HotMoE effectively utilizes the capabilities of different experts to obtain better overall performance. Compared with previous state-of-the-art hyperspectral object tracking networks, our model has significantly reduced inference time and performs well, with a processing speed of 43.7 FPS and an AUC of 0.704 with the HOT2022 dataset. The codes and models are available at https://github.com/supertyd/hotmoe.

Compared to conventional RGB images, hyperspectral images offer a more comprehensive range of spectral information, encompassing both visible and infrared bands. This enhanced spectral information facilitates trackers in effectively differentiating the target object from background clutter, realizing more robust recognition. However, hyperspectral video cameras exhibit variability in terms of spectral ranges and band numbers, producing distinct modalities. Developing specialized networks for each modality proves to be inefficient and time-consuming, while networks trained for one modality struggle to perform well on others, generating unsatisfactory outcomes. To confront these challenges encountered in various tracking scenarios, HyperTrack is proposed as a unified object tracking network tailored for hyperspectral videos. The proposed HyperTrack can be employed individually for single object tracking across three different modalities of hyperspectral videos: near infrared (NIR), visible (VIS), and red-to-near infrared (RedNIR). Specifically, since hyperspectral data have multiple bands, a band gate module is introduced into the network to enable it to select bands from hyperspectral images, thereby reducing the dimensionality for hyperspectral images. Furthermore, in order to effectively utilize the variability amongst the three different modal data, a computationally simple band embedding module is introduced to improve the object tracking performance of different modal hyperspectral videos. Additionally, a hybrid attention module is devised to efficiently extract and interact features between the template and search at each stage. As a unified network, HyperTrack achieves state-of-the-art comprehensive results across three different types of hyperspectral videos, particularly excelling with VIS and NIR type data. The code and models are publicly available at https://github.com/supertyd/HyperTrack.

Satellite-based aircraft monitoring is an important complement to ground surveillance systems, providing strong support for the safe, efficient, and reliable operation of global aviation. Most existing aircraft detection datasets are derived from still satellite imagery, making it difficult to detect flying aircraft. Although video satellite imagery can provide motion cues, its spatial coverage is limited, making it challenging to capture flying aircraft targets that are sparsely distributed over wide areas. Each Sentinel-2 satellite image covers a width of hundreds of kilometers, providing favorable conditions for monitoring flying aircraft. Beyond this, the physical design of its multispectral instruments (MSIs) induces parallax effects for moving objects in multispectral imagery, enabling a novel approach for the detection of flying aircraft. We construct a flying aircraft detection dataset (S2Aircraft) based on Sentinel-2 satellite multispectral imagery with a spatial resolution of 10 m. The dataset is annotated with oriented bounding boxes (OBBs) and includes both RGB and NIR spectral bands. Additionally, we design an efficient flying aircraft detection network (FADet), which maps input images to a high-dimensional nonlinear feature space while maintaining low computational complexity. Moreover, for single-class object detection tasks, the model employs a semi-decoupled head to achieve efficient detection. Finally, a loss function is specifically designed according to the geometric characteristics of targets in the S2Aircraft dataset, significantly improving the accuracy and stability of oriented object detection. Extensive experiments demonstrate the effectiveness and advancement of our FADet. Specifically, on our S2Aircraft dataset, FADet achieves competitive performance reaching 2.6 GFLOPs and 96.3% mAP$_{50}$ on the S2Aircraft dataset. On two public datasets, HRSC2016 and CORS-ADD, FADet achieves mAP$_{50}$ of 90.90% and 94.16%, respectively. The dataset and codes are available at https://github.com/maonanmaonan/FADet.

Multispectral filter array (MSFA) imaging enables low-cost, real-time, and compact spectral sensing, making it highly suitable for IoT applications such as smartphone imaging, smart agriculture, UAV-based remote sensing, and wearable diagnostics. However, due to the strong coupling among spectral response functions (SRFs), MSFA pattern, and reconstruction algorithms, achieving high-fidelity reconstruction remains a significant challenge. Conventional independent optimization of each module often leads to suboptimal results, highlighting the necessity of a co-optimized design method. To this end, we propose an end-to-end spectral reconstruction framework that jointly optimizes the SRF, MSFA pattern, and reconstruction network. In our method, the SRFs are parameterized as Gaussian functions defined by the center wavelength and full width at half maximum (FWHM). The MSFA pattern is designed using the optimal sphere-packing (OSP) method to enhance sampling uniformity and reduce artifacts. In addition, a dual-stage reconstruction pipeline is introduced: the first stage employs a Res2FFT-based U-Net with multi-scale gated attention (MGA) for spatial demosaicing, and the second stage uses an enhanced deep residual network (EDSR) for spectral super-resolution. All components are jointly trained under a hybrid loss with supervision from both multispectral and hyperspectral reconstruction. Experiments on three benchmark datasets demonstrate that our method achieves superior spatial and spectral fidelity compared to state-of-the-art methods. Furthermore, the model has been successfully deployed on an Nvidia Jetson-based IoT device, achieving real-time inference at 45 ms per frame, validating its potential for real-time spectral imaging in IoT systems.

A multispectral filter array (MSFA) is a typical snapshot spectral imaging technique, but a single sensor is limited by its narrow overall spectral range and limited number of bands. To obtain a wider spectral range and more imaging bands, the use of multiple MSFA sensors has become an effective solution. However, the existing demosaicing algorithms were designed mainly for single-MSFA systems and lack effective methods for jointly reconstructing MSFA images that are synchronously acquired from multiple sources. To address this issue, we propose a novel multi-MSFA demosaicing network, the KAN-enhanced siamese transformer pyramid network (STP-KAN). This network is built on a siamese feature pyramid architecture and uses a Kolmogorov–Arnold Network (KAN)-based half-shuffle multi-head self-attention as its basic module, effectively leveraging the long-range modeling capabilities of transformers and the powerful nonlinear representation ability of the KAN. A cross-attention mechanism is utilized to efficiently share features among multi-MSFA images. A spatial-spectral attention mechanism is embedded into the skip connections of the network to fuse deep spatial and spectral features. MSFA demosaicing experiments conducted on the ARAD-1K open-source natural scene dataset and our self-built AVIRIS-NG remote sensing dataset demonstrate the superior performance of our proposed method. The source code is available at https://github.com/Stephen-future/STP-KAN.

Multispectral images contain rich spatial details and spectral information, but their large data volume places heavy demands on transmission bandwidth and storage resources. Existing multispectral image compression methods still face challenges in jointly modeling spatial and inter-spectral redundancies. In particular, similar feature-processing strategies adopted in some encoder–decoder designs, limited spectral–spatial interaction, and inadequate multi-scale modeling limit further improvements in compression efficiency. To address these issues, this paper proposes a spectral–spatial decoupling and fusion network with multi-scale perception (SSDFN) for multispectral image compression. SSDFN is built on an end-to-end learned compression framework and introduces differentiated spectral- and spatial-oriented processing pathways with tailored fusion mechanisms for the encoder and decoder. The encoder focuses on redundancy suppression and compact latent representation learning, whereas the decoder emphasizes feature interaction, information compensation, and high-fidelity reconstruction. In addition, a Multi-Scale Feature Fusion Block is developed to model complex spatial structures in multispectral remote sensing scenes. Hyperprior and context entropy models are further incorporated to improve entropy estimation. Experiments on the Landsat-8 and Sentinel-2 datasets show that SSDFN achieves competitive rate–distortion performance and provides consistent improvements in spatial reconstruction quality, perceptual quality, and spectral fidelity.