Cross-scene hyperspectral image (HSI) classification has received widespread attention due to its practicality. However, domain adaptation (DA)-based cross-scene HSI classification methods are typically tailored for a specific target scene involved in model training and require retraining for new scenes. We instead propose a novel spectral–spatial adversarial multidomain synthetic network (S2AMSnet) that can be trained on a single source domain (SD) and generalized to unseen domains. S2AMSnet improves the robustness of the model to the unseen domain by expanding the diverse distribution of the SD. Specifically, to spatially and spectrally generate diversified generative domain (GD), the spectral–spatial domain-generation network (S2DGN) is designed, and two S2DGNs with the same structure but not shared parameters are enabled to generate diversified GD through two-step min–max strategy. A Multidomain mixing module is employed to expand the diversity of the GD further and enhance their class-domain semantic consistency information. In addition, a multiscale mutual information (MI) regularization network (MMIRN) is used to constrain the S2DGN so that the intrinsic class semantic information of its generated GD does not deviate from the SD. A semantic consistency discriminator with spectral–spatial feature extraction capability is utilized to capture class-domain semantic consistency information from diverse GD to obtain cross-domain invariant knowledge. Comparative analysis with eight state-of-the-art transfer learning methods on three real HSI datasets, along with an ablation study, validates the effectiveness of the proposed S2AMSnet in the cross-scene HSI classification task. The codes of this work will be available at https://github.com/daxichen/S2AMSnet .

Cross-spectral local feature matching between visual and thermal images benefits many vision tasks in low-light environments, including image-to-image fusion and camera re-localization. An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of visible–thermal matching is the availability of large-scale and high-quality annotated datasets. However, publicly available datasets are either in relative small quantity scales or have limited pose annotations due to the expensive cost of data acquisition and annotation, which severely hinders the development of this field. In this paper, we proposed a multi-view thermal–visible image dataset for large-scale cross-spectral matching. We first recovered a 3D reference model from a group of collected RGB images, in which a certain image (bridge) shares almost the same pose as the thermal query. We then effectively registered the thermal image to the model based on manually annotating a 2D-2D tie point between the bridge and the thermal. In this way, through simply annotating one same viewpoint image pair, numerous overlapping image pairs between thermal and visible could be available. We also proposed a semi-automatic approach for generating accurate supervision for training multi-view cross-spectral matching. Specifically, our dataset consists of 40,644 cross-modal pairs with well supervision, covering multiple complex scenes. In addition, we also provided the camera metadata, 3D reference model, depth map of the visible images and 6-DoF pose of all images. We extensively evaluated the performance of state-of-the-art algorithms on our dataset and provided a comprehensive analysis of the results. We will publish our dataset and pre-processing code.

Thermal infrared (TIR) images capture temperature in a non-invasive manner, making them valuable for generating 3D models that reflect the spatial distribution of thermal properties within a scene. Current TIR image-based 3D reconstruction methods primarily focus on static conditions, which only capture the spatial distribution of thermal radiation but lack the ability to represent its temporal dynamics. The absence of dedicated datasets and effective methods for dynamic 3D representation are two key challenges that hinder progress in this field. To address these challenges, we propose a novel dynamic thermal 3D reconstruction method, named ThermalGS, based on 3D Gaussian Splatting (3DGS). ThermalGS employs a data-driven approach to directly learn both scene structure and dynamic thermal representation, using RGB and TIR images as input. The position, orientation, and scale of Gaussian primitives are guided by the RGB mesh. We introduce feature encoding and embedding networks to integrate semantic and temporal information into the Gaussian primitives, allowing them to capture dynamic thermal radiation characteristics. Moreover, we construct the Thermal Scene Day-and-Night (TSDN) dataset, which includes multi-view, high-resolution aerial RGB reference images and TIR images captured at five different times throughout the day and night, providing a benchmark for dynamic thermal 3D reconstruction tasks. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on the TSDN dataset, with an average absolute temperature error of 1 °C and the ability to predict surface temperature variations over time.

With the continuous advancement of the construction of smart cities, the availability of large-scale and semantically enriched datasets is essential for enhancing the machine’s ability to understand urban scenes. Mesh data have a distinct advantage over point cloud data for large-scale scenes, as they can provide inherent geometric topology information and consume less memory space. However, existing publicly available large-scale scene mesh datasets are limited in scale and semantic richness and do not cover a wide range of urban semantic information. The development of 3D semantic segmentation algorithms depends on the availability of datasets. Moreover, existing large-scale 3D datasets lack various types of official annotation data, which hinders the widespread applicability of benchmark applications and may cause label errors during data conversion. To address these issues, we present a comprehensive urban-scale semantic segmentation benchmark dataset. It is suitable for various research pursuits on semantic segmentation methodologies. This dataset contains finely annotated point cloud and mesh data types for 3D, as well as high-resolution original 2D images with detailed 2D semantic annotations. It is constructed from a 3D reconstruction of 10,840 UVA aerial images and spans a vast area of approximately 2.85 square kilometers that covers both urban and rural scenes. The dataset is composed of 152,298,756 3D points and 289,404,088 triangles. Each 3D point, triangular mesh, and the original 2D image in the dataset are carefully labeled with one of the ten semantic categories. Six typical 3D semantic segmentation methods were compared on the CUS3D dataset, with KPConv demonstrating the highest overall performance. The mIoU is 59.72%, OA is 89.42%, and mAcc is 97.88%. Furthermore, the experimental results on the impact of color information on semantic segmentation suggest that incorporating both coordinate and color features can enhance the performance of semantic segmentation. The current limitations of the CUS3D dataset, particularly in class imbalance, will be the primary target for future dataset enhancements.

While visual localization has made significant advances in recent years, it still lacks robustness in low-light situations. Thermal camera images, which capture temperature data, provide a potential solution for these environments. However, the scarcity of well-annotated, publicly available datasets for thermal localization, particularly those focused on absolute pose estimation, impedes further advancement in this field. In this research, we introduce a novel dataset that includes six-degree-of-freedom (6-DoF) absolute poses of query images for large-scale, realistic aerial localization of thermal images. Besides, we introduce a render-to-localization pipeline tailored for thermal image localization. This pipeline predicts the 6-DoF pose of a query using a synthetic technique based on geometric refined thermal model. Experimental results demonstrate the effectiveness of our method on this newly proposed dataset. Notably, our method achieves a median position error of less than 1.5 m and a median angle error of less than 1.5° under diverse test conditions. A comprehensive analysis of factors influencing localization accuracy is also provided. Our code and dataset will be available at https://github.com/RingoWRW/ATLoc .