Passive non-line-of-sight (NLOS) imaging has potential applications in autonomous driving and search and rescue, but current deep learning approaches often produce suboptimal images due to sparse and homogeneous projection features, leading to an ill-posed reconstruction process. To address this, we propose the Hyperspectral Fusion NLOS imaging technique (HFN-Net), which first leverages high-dimensional features from multiple spectra and incorporates spatial-spectral attention from a hyperspectral full-color auto-encoder. This method improves color fidelity and structural details by fully utilizing the limited information and increasing feature diversity. Additionally, we developed the Hyperspectral NLOS dataset (HS-NLOS) for training and evaluation. Experimental results show that HFN-Net offers performance improvements over traditional passive NLOS 2D imaging techniques, emphasizing the importance of multi-spectral information.
1 July 2024 in Journal of Computer Science and Technology
In environments where vision-based depth estimation systems, such as those utilizing infrared or imaging technologies, encounter limitations—particularly in low-light conditions—alternative approaches become essential. Echo depth estimation emerges as a compelling solution by leveraging the time delay of echoes to map the geometric structure of the surrounding environment. This method offers distinct advantages in specific scenarios, providing reliable data for accurate scene understanding and 3D reconstruction. Traditional echo depth estimation techniques primarily depend on spatial information captured by the encoder and depth predictions made by the decoder. However, these methods often fail to fully exploit the rich depth features present at different simultaneous frequencies. To address this challenge, we propose an echo depth estimation method via Attention based Hierarchical Multi-scale Feature Fusion Network (AHMF-Net). This network is designed to extract spatial depth information from echo spectrograms across multiple scales and hierarchical levels, while fusing the most relevant information using an attention mechanism. AHMF-Net introduces two key modules in hierarchical levels: the Intra-layer Multi-scale Attention Feature Fusion (IMAF) module, which functions as the encoder to capture multi-scale features across varying granularities, and the Inter-layer Multi-Scale Detail Feature Fusion (IMDF) module, which integrates features from all encoding layers into the decoder to enable effective inter-layer multi-scale fusion. Additionally, the encoder incorporates an attention mechanism that enhances depth-related features by capturing channel dependencies at multiple scales. We evaluated AHMF-Net on the Replica, Matterport3D, and BatVision datasets, where it consistently outperformed state-of-the-art models in echo-based depth estimation, demonstrating superior accuracy and robustness. The source code is publicly available at https://github.com/wjzhang-ai/AHMF-Net.