In the field of hyperspectral video tracking (HVT), occclusion poses a challenging issue without a satisfactory solution. To address this challenge, the current study explores the application of capsule networks in HVT and proposes an approach based on octave convolution and a spatial–spectral capsule network (OCSCNet). Specifically, the spatial–spectral octave convolution module is designed to learn features from hyperspectral images by integrating spatial and spectral information. Hence, unlike traditional convolution, which is limited to learning spatial features, the proposed strategy also focuses on learning and modeling the spectral features. The proposed spatial–spectral capsule network integrates spectral information to distinguish among underlying capsule categories based on their spectral similarity. The approach enhances separability and establishes relationships between different components and targets at various scales. Finally, a confidence threshold judgment module utilizes the information from the initial and adjacent frames for relocating the lost target. Experiments conducted on the HOT2023 dataset illustrate that the proposed model outperforms state-of-the-art methods, achieving a success rate of 65.2% and a precision of 89.3%. In addition, extensive experimental results and visualizations further demonstrate the effectiveness and interpretability of the proposed OCSCNet.

In this paper, we propose a novel city-scale distance sensing algorithm based on atmosphere optics. The suspended particles, especially in bad weather, would attenuate the light at almost all wavelengths. Observing this fact and starting from the light scattering mechanism, we derive a bispectral distance sensing algorithm by leveraging the difference of extinction coefficient between two specifically selected near infrared wavelengths. The extinction coefficient of the atmosphere is related to both wavelength and meteorological conditions, also known as visibility, such as the fog and haze day. To account for different bad weather conditions, we explicitly introduce visibility into our algorithm by incorporating it into the calculation of extinction coefficient, making our algorithm simple yet effective. To capture the data, we build a bispectral imaging system that is able to take a pair of images with a monochrome camera and two narrow band-pass filters. We also present a wavelength selection strategy that allows us to accurately sense distance regardless of material reflectance and texture. Specifically, this strategy determines two distinct near infrared wavelengths by maximising the extinction coefficient difference while minimizing the influence of building’s reflectance variance. The experiments empirically validate our model and its practical performance on the distance sensing for the city-scale buildings.

This paper introduces a novel depth recovery method based on light absorption in water. Water absorbs light at almost all wavelengths whose absorption coefficient is related to the wavelength. Based on the Beer-Lambert model, we introduce a bispectral depth recovery method that leverages the light absorption difference between two near-infrared wavelengths captured with a distant point source and orthographic cameras. Through extensive analysis, we show that accurate depth can be recovered irrespective of the surface texture and reflectance, and introduce algorithms to correct for nonidealities of a practical implementation including tilted light source and camera placement, nonideal bandpass filters and the perspective effect of the camera with a diverging point light source. We construct a coaxial bispectral depth imaging system using low-cost off-the-shelf hardware and demonstrate its use for recovering the shapes of complex and dynamic objects in water. We also present a trispectral variant to further improve robustness to extremely challenging surface reflectance. Experimental results validate the theory and practical implementation of this novel depth recovery paradigm, which we refer to as shape from water.