Transformers have shown remarkable success in modeling sequential data and capturing intricate patterns over long distances. Their self-attention mechanism allows for efficient parallel processing and scalability, making them well-suited for the high-dimensional data in hyperspectral and LiDAR imagery. However, further research is needed on how to more deeply integrate the features of two modalities in attention mechanisms. In this paper, we propose a novel Multi-Feature Cross Attention-Induced Transformer Network (MCAITN) designed to enhance the classification accuracy of hyperspectral and LiDAR data. The MCAITN integrates the strengths of both data modalities by leveraging a cross-attention mechanism that effectively captures the complementary information between hyperspectral and LiDAR features. By utilizing a transformer-based architecture, the network is capable of learning complex spatial-spectral relationships and long-range dependencies. The cross-attention module facilitates the fusion of multi-source data, improving the network’s ability to discriminate between different land cover types. Extensive experiments conducted on benchmark datasets demonstrate that the MCAITN outperforms state-of-the-art methods in terms of classification accuracy and robustness.
Objective: We propose a deep-learning-based underwater target detection system that can effectively solve the problem of underwater optical image target detection and recognition. Methods: In this paper, based on the depth of the underwater optical image target detection and recognition and using a learning model, we put forward corresponding solutions using the concept of style migration solutions, such as training samples. A lack of variability and poor generalization of practical applications presents a challenge for underwater object identification. The UW_YOLOv3 lightweight model was proposed to solve the problems of calculating energy consumption and storage resource limitations in underwater application scenarios. The detection and recognition module, based on deep learning, can deal with the degradation process of underwater imaging by embedding an image enhancement module into the detection and recognition module for the joint tuning and transferring of knowledge. Results: The detection accuracy of the UW_YOLOv3 model designed in this paper outperformed the lightweight algorithm YOLOV3-TINY by 7.9% at the same image scale input. Compared with other large algorithms, the detection accuracy was lower, but the detection speed was much higher. Compared with the SSD algorithm, the detection accuracy was only 4.7 lower; the speed was 40.9 FPS higher; and the rate was nearly 16 times higher than Faster R-CNN. When the input scale was 224, although part of the accuracy was lost, the detection speed doubled, reaching 156.9 FPS. Conclusion: Based on our framework, the problem of underwater optical image target detection and recognition can be effectively solved. Relevant studies have not only enriched the theory of target detection and glory, but have also provided optical glasses with a clear vision for appropriate underwater application systems.
18 April 2022 in Computer Methods and Programs in Biomedicine
We present a novel view synthesis method that introduces radial field representation of density and tidal flat appearance in neural rendering. Our method aims to generate realistic images from new viewpoints by using continuous scene information generated from different sampling points along a set of identical rays. This approach significantly improves rendering quality and reduces blurring and aliasing artifacts compared to existing techniques such as Nerfacto. Our model employs the spherical harmonic function to efficiently encode viewpoint orientation information and integrates image features from neighboring viewpoints for enhanced fusion. This results in an accurate and detailed reconstruction of the scene's geometry and appearance. We evaluate our approach on publicly available datasets containing a variety of indoor and outdoor scenes, as well as on customized tidal flats datasets. The results show that our algorithm outperforms Nerfacto in terms of PSNR (peak signal-to-noise ratio), SSIM (structural similarity index measure), and LPIPS (learned perceptual image patch similarity) metrics, demonstrating superior performance in both complex and simple environments. This study emphasizes the potential of our approach in advancing view synthesis techniques and provides a powerful tool for environmental research and conservation efforts in dynamic ecosystems such as mudflats. Future work will focus on further optimizations and extensions to improve the efficiency and quality of the rendering process.
Infrared and visible images provide complementary cues for maritime ship detection, but practical dual-sensor systems often produce image pairs that are not strictly registered. Directly fusing such unregistered pairs may introduce ghosting artifacts, blurred target boundaries, and feature conflicts, especially over weak-texture sea surfaces where reliable correspondence cues are sparse. To address this problem, we propose UA-FusionDet, an unregistered-aware infrared-visible fusion framework with cross-modal feature alignment for maritime ship detection. The proposed framework extracts visible and infrared features with a dual-branch encoder, aligns the visible feature to the infrared reference through cross-modal deformable feature alignment, and suppresses unstable background offsets using sea-surface saliency guidance. Wavelet-guided complementary fusion then decomposes the aligned features into low- and high-frequency sub-bands, enabling frequency-aware fusion of infrared thermal saliency and visible structural details before feeding the fused representation to both a lightweight reconstruction decoder and a ship detection head. The reconstruction decoder provides auxiliary image-level regularization, while the detection branch supervises the task-oriented fused representation with ship bounding-box annotations. UA-FusionDet does not require registration ground truth or fused-image ground truth during training, making it suitable for realistic maritime monitoring scenarios with imperfectly aligned visible and infrared sensors. Experiments on 3132 unregistered visible–LWIR maritime image pairs show that UA-FusionDet achieves a precision of 0.904, a recall of 0.866, an of 0.912, and an of 0.566, exceeding the strongest competing method by 2.5 and 2.4 percentage points on the two mAP metrics, respectively, while maintaining an inference speed of 52.6 FPS. These results demonstrate that the proposed alignment and fusion framework improves detection accuracy under cross-modal misregistration while retaining practical inference efficiency.