In the field of hyperspectral video tracking (HVT), occclusion poses a challenging issue without a satisfactory solution. To address this challenge, the current study explores the application of capsule networks in HVT and proposes an approach based on octave convolution and a spatial–spectral capsule network (OCSCNet). Specifically, the spatial–spectral octave convolution module is designed to learn features from hyperspectral images by integrating spatial and spectral information. Hence, unlike traditional convolution, which is limited to learning spatial features, the proposed strategy also focuses on learning and modeling the spectral features. The proposed spatial–spectral capsule network integrates spectral information to distinguish among underlying capsule categories based on their spectral similarity. The approach enhances separability and establishes relationships between different components and targets at various scales. Finally, a confidence threshold judgment module utilizes the information from the initial and adjacent frames for relocating the lost target. Experiments conducted on the HOT2023 dataset illustrate that the proposed model outperforms state-of-the-art methods, achieving a success rate of 65.2% and a precision of 89.3%. In addition, extensive experimental results and visualizations further demonstrate the effectiveness and interpretability of the proposed OCSCNet.
16 October 2023 in IEEE Geoscience and Remote Sensing Letters
In order to effectively utilize spectral and object spatial information to improve tracking performance, we design an Hyperspectral Video (HSV) tracker, namely DSP-Net, to integrate the various prior information. The gradient difference between spectral vectors is explored to develop a clustering technique. The approach generates a binary mask containing target spectral information and appearance clues. A feature cache is introduced to store historical information. Additionally, the channel shift operation is used on the timing to capture the trajectory clues of the target. With the help of the non-local mechanism, the trajectory clues, appearance clues and spectral information of the target are finally integrated, using the designed spectral-spatial joint perception module to enhance the expression of the target. Experimental results show that DSP-Net outperforms state-of-the-art HSV trackers on existing dataset.
A magnetic field sensing system based on a phase-shift fiber loop ring-down (FLRD) technique and multi-mode interferometer (MI) coated with magnetic fluid (MF) is proposed and demonstrated. The MI is constructed by splicing a segment of no-core fiber between two sections of single-mode fibers, which is then immersed in MF to serve as a sensing head with the advantages of simple fabrication and specific magnetic sensitivity. Due to the magnetic refractive index tunable properties of the MF, the magnetic-field-dependent loss will be introduced in the fiber loop by the sensing head. Such magnetic-induced loss would be accumulated during the round trip of the optical carrier and reflected on the phase information of the modulated signal. The phase-shift changes with the applied magnetic field strength, enabling magnetic field sensing through phase-shift measurements. The sensing system is experimentally demonstrated and a sensitivity of 0.704×10−3deg/Gs in the linear region is achieved. Moreover, the stability and repeatability of the system are verified, leading to a promising method for magnetic field measurements.
In order to address the problems of in-plane rotation and fast motion during near-infrared (NIR) video target tracking, this study explores the application of capsule networks in NIR video and proposes a capsule network method based on background information and spectral position prediction. First, the history frame background information extraction module is proposed. This module performs spectral matching on the history frame images through the average spectral curve of the groundtruth value of the target and makes a rough distinction between the target and the background. On this basis, the background information of history frames is stored as a background pool for subsequent operations. The proposed background target routing module combines the traditional capsule network algorithm with spectral information. Specifically, the similarity between the target capsule and the background capsule in the spectral feature space is calculated, and the capsule weight allocation mechanism is dynamically adjusted. Thus, the discriminative ability of the target and background is strengthened. Finally, the spectral information position prediction module locates the center of the search region in the next frame by fusing the position information and spectral features of adjacent frames with the current frame. This module effectively reduces the computational complexity of feature extraction by capsule networks and improves tracking stability. Experimental evaluations demonstrate that the novel framework achieves superior performance compared to current methods, attaining a 70.3% success rate and 88.4% accuracy on near-infrared (NIR) data. Meanwhile, for visible spectrum (VIS) data analysis, the architecture maintains competitive effectiveness with a 59.6% success rate and 78.8% precision.