10 February 2023 in IEEE Geoscience and Remote Sensing Letters
Underwater terrain image (UTI) matching provides a new idea for terrain-aided localization technology. However, the diversity of UTIs makes it very difficult to accurately match real-time terrain images with reference terrain images. To improve the localization accuracy of UTIs, this letter describes how UTI matching can be modeled as a deep learning problem based on supervised algorithms. Specifically, we propose a hierarchical matching strategy for the diverse features of UTIs by combining the deep and shallow feature characteristics of a convolutional neural network to improve the localization accuracy. We also propose a sample-generation strategy based on candidate area quality index classification, and develop a data enhancement method by simulating the underwater terrain environment to overcome the lack of network training data. Simulations prove that the proposed method achieves accurate and stable localization results.
27 October 2015 in Journal of Marine Science and Application
16 November 2022 in IEEE Transactions on Geoscience and Remote Sensing
The distinctive nature of the intrinsic features of underwater terrain images, along with their variability of intensity and texture due to the complex environment in which they are acquired, poses great challenges to manual feature-based template-matching methods. In this paper, we propose a novel data-driven framework for template matching of underwater terrain images. This combines the idea of contrastive learning with the template-matching process, allowing self-supervised end-to-end training without the need for additional data annotation. The resulting positive and negative sample comparisons improve the discriminatory ability of the model. Specifically, we achieve data enhancement by simulating grayscale and texture differences between images to improve the anti-interference capability of the model, and we propose a patch-based sample-extraction strategy to extract positive and negative samples. To further improve the discriminative ability for features among patches, we propose a feature-fusion module based on an attention mechanism. This improves the level of discrimination of the features to be matched by fusing the contextual features of patches. In addition, we solve the problem of matching multi-scale templates by introducing a masking mechanism. We performed detailed ablation experiments on the components in the framework to verify their effectiveness. We also compared the performance of our model with other state-of-the-art template-matching methods. The experimental results demonstrate that our proposed method has superior matching performance when compared to its competitors.
4 October 2022 in IEEE Sensors Journal
Since the existing forward-looking sonar image mosaic algorithm is completed in the image-domain with reference to the optical image, and the characteristics of forward-looking sonar imaging are not fully considered, obvious edge-seams and pseudo-target values are prone to generate after mosaic, resulting in poor visual effect and low operation efficiency. To avoid the aforementioned problems, the mosaic operation of beam-domain forward-looking sonar images is proposed. Combined with the imaging principle of forward-looking sonar, this study takes advantage of the variation of the sampling beams and sampling points of the same target point during sonar movement to analyze the deformation law of the target in beam-domain which is scanned in image-domain, and then proposes the concept of expression domain mapping model. Considering the need of aligning acoustic origin, the target is deformed according to this model in the process of beam-domain mosaic, and the target alignment is completed. Subsequently, the Laplace pyramid layered algorithm is used for fusion and mosaic, and finally mapped to image-domain for display. Experimental results illustrate that this kind of algorithm can remove the mosaic edge-seams generated during mosaicking in the image-domain because it uses the sonar raw image data for mosaic in the beam-domain and avoids the appearance of sector edges in the image-domain; The actual pixel points sampled by sonar are used for mosaic, which effectively eliminates the pseudo-target value generated by the mosaic of the inter-beam interpolation in the image-domain; Moreover, the number of image pixels in the beam-domain is small, and the mosaic time consuming of adjacent frames and continuous frames is more than half shorter than that of the image domain, which significantly improves the mosaic efficiency and has good mosaic performance.
3 March 2023 in IEEE Signal Processing Letters
To solve the rotational changes in matching localization of an underwater terrain image, this letter proposes the ring-masked attention network (RMANet), a model-driven deep network for rotational template-matching tasks. Since traditional convolutional neural networks cannot effectively encode rotational changes, we introduce a rotation-equivariant network to extract the rotation-equivariant features. This network determines the rotation of an image at the pixel level. Based on the rotation-equivariant features, we propose the ring-masked attention module (RMAM), which combines the idea of the ring projection transform with an attention mechanism to extract the rotation-invariant features that are independent of the orientation. The overall model combines the rotation-equivariant network with RMAM into an end-to-end network that can exploit both the feature-representation capability of the learning-based model and domain knowledge. Our experimental results show that, compared with popular approaches targeting rotational matching tasks, RMANet achieves performance gains in terms of both matching accuracy and running speed.