Cross-spectral local feature matching between visual and thermal images benefits many vision tasks in low-light environments, including image-to-image fusion and camera re-localization. An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of visible–thermal matching is the availability of large-scale and high-quality annotated datasets. However, publicly available datasets are either in relative small quantity scales or have limited pose annotations due to the expensive cost of data acquisition and annotation, which severely hinders the development of this field. In this paper, we proposed a multi-view thermal–visible image dataset for large-scale cross-spectral matching. We first recovered a 3D reference model from a group of collected RGB images, in which a certain image (bridge) shares almost the same pose as the thermal query. We then effectively registered the thermal image to the model based on manually annotating a 2D-2D tie point between the bridge and the thermal. In this way, through simply annotating one same viewpoint image pair, numerous overlapping image pairs between thermal and visible could be available. We also proposed a semi-automatic approach for generating accurate supervision for training multi-view cross-spectral matching. Specifically, our dataset consists of 40,644 cross-modal pairs with well supervision, covering multiple complex scenes. In addition, we also provided the camera metadata, 3D reference model, depth map of the visible images and 6-DoF pose of all images. We extensively evaluated the performance of state-of-the-art algorithms on our dataset and provided a comprehensive analysis of the results. We will publish our dataset and pre-processing code.

Thermal infrared (TIR) images capture temperature in a non-invasive manner, making them valuable for generating 3D models that reflect the spatial distribution of thermal properties within a scene. Current TIR image-based 3D reconstruction methods primarily focus on static conditions, which only capture the spatial distribution of thermal radiation but lack the ability to represent its temporal dynamics. The absence of dedicated datasets and effective methods for dynamic 3D representation are two key challenges that hinder progress in this field. To address these challenges, we propose a novel dynamic thermal 3D reconstruction method, named ThermalGS, based on 3D Gaussian Splatting (3DGS). ThermalGS employs a data-driven approach to directly learn both scene structure and dynamic thermal representation, using RGB and TIR images as input. The position, orientation, and scale of Gaussian primitives are guided by the RGB mesh. We introduce feature encoding and embedding networks to integrate semantic and temporal information into the Gaussian primitives, allowing them to capture dynamic thermal radiation characteristics. Moreover, we construct the Thermal Scene Day-and-Night (TSDN) dataset, which includes multi-view, high-resolution aerial RGB reference images and TIR images captured at five different times throughout the day and night, providing a benchmark for dynamic thermal 3D reconstruction tasks. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on the TSDN dataset, with an average absolute temperature error of 1 °C and the ability to predict surface temperature variations over time.

With the continuous advancement of the construction of smart cities, the availability of large-scale and semantically enriched datasets is essential for enhancing the machine’s ability to understand urban scenes. Mesh data have a distinct advantage over point cloud data for large-scale scenes, as they can provide inherent geometric topology information and consume less memory space. However, existing publicly available large-scale scene mesh datasets are limited in scale and semantic richness and do not cover a wide range of urban semantic information. The development of 3D semantic segmentation algorithms depends on the availability of datasets. Moreover, existing large-scale 3D datasets lack various types of official annotation data, which hinders the widespread applicability of benchmark applications and may cause label errors during data conversion. To address these issues, we present a comprehensive urban-scale semantic segmentation benchmark dataset. It is suitable for various research pursuits on semantic segmentation methodologies. This dataset contains finely annotated point cloud and mesh data types for 3D, as well as high-resolution original 2D images with detailed 2D semantic annotations. It is constructed from a 3D reconstruction of 10,840 UVA aerial images and spans a vast area of approximately 2.85 square kilometers that covers both urban and rural scenes. The dataset is composed of 152,298,756 3D points and 289,404,088 triangles. Each 3D point, triangular mesh, and the original 2D image in the dataset are carefully labeled with one of the ten semantic categories. Six typical 3D semantic segmentation methods were compared on the CUS3D dataset, with KPConv demonstrating the highest overall performance. The mIoU is 59.72%, OA is 89.42%, and mAcc is 97.88%. Furthermore, the experimental results on the impact of color information on semantic segmentation suggest that incorporating both coordinate and color features can enhance the performance of semantic segmentation. The current limitations of the CUS3D dataset, particularly in class imbalance, will be the primary target for future dataset enhancements.

While visual localization has made significant advances in recent years, it still lacks robustness in low-light situations. Thermal camera images, which capture temperature data, provide a potential solution for these environments. However, the scarcity of well-annotated, publicly available datasets for thermal localization, particularly those focused on absolute pose estimation, impedes further advancement in this field. In this research, we introduce a novel dataset that includes six-degree-of-freedom (6-DoF) absolute poses of query images for large-scale, realistic aerial localization of thermal images. Besides, we introduce a render-to-localization pipeline tailored for thermal image localization. This pipeline predicts the 6-DoF pose of a query using a synthetic technique based on geometric refined thermal model. Experimental results demonstrate the effectiveness of our method on this newly proposed dataset. Notably, our method achieves a median position error of less than 1.5 m and a median angle error of less than 1.5° under diverse test conditions. A comprehensive analysis of factors influencing localization accuracy is also provided. Our code and dataset will be available at https://github.com/RingoWRW/ATLoc .

Leveraging natural language to guide visual models for single-domain generalization (SDG) in hyperspectral image classification (HSIC) is a promising yet challenging frontier. A fundamental obstacle is the inherent heterogeneity between visual and linguistic modalities, where modality-specific attributes interfere with direct feature alignment, consequently limiting the generalization capacity of the hyperspectral encoder. To overcome this, we propose a novel framework, Decoupling and Aligning Modality-Shared Semantics (DAMS). The core innovation of DAMS is a feature disentanglement strategy that isolates modality-shared semantics from modality-specific attributes for both spectral and textual data. This allows for a more precise and robust alignment of the shared components in a common embedding space via modality-shared supervised contrastive learning. This core alignment is further regularized by two auxiliary objectives: a bidirectional cross-modal transformation task to enforce semantic consistency, and an intra-modal contrastive loss to ensure feature invariance across data augmentations. Extensive experiments on three challenging cross-domain HSIC benchmarks validate our approach, demonstrating that DAMS establishes a new state-of-the-art by significantly outperforming existing methods. Ablation studies further confirm that our disentangle-and-align paradigm is the primary driver of the model's superior generalization performance. The code is available at https://github.com/daxichen/DAMS.

Single-source domain generalization (DG) presents a formidable challenge in cross-scene hyperspectral image (HSI) classification, as models trained on a single domain often struggle to generalize due to spectral shifts. Prevailing approaches predominantly concentrate on expanding the source distribution through data augmentation or adversarial training techniques. However, such strategies often neglect a critical pitfall: in the absence of explicit constraints, models are prone to overfitting domain-specific spurious correlations rather than capturing intrinsic invariant semantics. Moreover, the reliance on adversarial training frequently introduces instability and convergence difficulties. To address these limitations, this paper proposes Cross-SPECL, a unified non-adversarial framework that transitions the paradigm to a structured “Stabilize-then-Disentangle” process. Specifically, the proposed framework first introduces a Spectral Patch Low-Frequency Transformation Network as a feature stabilizer. This module utilizes a dual-branch architecture wherein an auxiliary branch applies controlled perturbations to low-frequency components, thereby compelling the main branch to extract features resilient to style variations. To further consolidate this robustness, a Synergistic Feature Stabilization mechanism integrates Supervised and Prototype Contrastive Learning to enforce invariance to these perturbations and construct highly representative class-conditional prototypes. Building upon this foundation, the framework learns a Domain-Agnostic Directed Acyclic Graph (DAG) derived from these prototypes to explicitly uncover the causal structure. This process generates a causal mask that effectively prunes domain-specific spurious features, ensuring robust generalization. Extensive experimental evaluations on three cross-scene HSI classification benchmarks demonstrate that Cross-SPECL yields significant performance improvements over state-of-the-art methods.