27 January 2025 in IEEE Transactions on Geoscience and Remote Sensing
With the extensive use of multi-sensors in Unmanned Aerial Vehicles (UAVs), multi-modality information processing has become the research focus. In academic research pertaining to object detection and tracking tasks in unmanned aerial vehicles, researchers often align visible-infrared image pairs as a preprocessing step. However, in actual tasks, the dual-modality image pair acquired by unmanned aerial vehicles is unaligned, which significantly limits the application of downstream tasks. At present, there are no publicly available multi-modality image alignment datasets for unmanned aerial vehicles. In this paper, we present a large-scale benchmark for the dual-modality image alignment task in unmanned aerial vehicles, including 81,000 training image pairs and 15,000 testing image pairs. Meanwhile, we propose a transformer-based dual-modality image alignment network as the baseline for this benchmark. Firstly, the algorithm extracts multi-scale features for image representation to address unaligned image pairs with varying resolutions. Secondly, a transformer-based alignment network is proposed to improve the fusion of features from heterogeneous modalities. Finally, deformable attention is adopted to alleviate the problem of memory explosion. Numerous experiments on this dual-modality image alignment benchmark are conducted to demonstrate the effectiveness of our algorithm. Source codes are available at https://github.com/gaozhinanjiu/UAVmatch.