Color plays an important role in human visual perception, reflecting the spectrum of objects. However, the existing infrared and visible image fusion methods rarely explore how to handle multi-spectral/channel data directly and achieve high color fidelity. This paper addresses the above issue by proposing a novel method with diffusion models, termed as Dif-Fusion, to generate the distribution of the multi-channel input data, which increases the ability of multi-source information aggregation and the fidelity of colors. In specific, instead of converting multi-channel images into single-channel data in existing fusion methods, we create the multi-channel data distribution with a denoising network in a latent space with forward and reverse diffusion process. Then, we use the the denoising network to extract the multi-channel diffusion features with both visible and infrared information. Finally, we feed the multi-channel diffusion features to the multi-channel fusion module to directly generate the three-channel fused image. To retain the texture and intensity information, we propose multi-channel gradient loss and intensity loss. Along with the current evaluation metrics for measuring texture and intensity fidelity, we introduce Delta E as a new evaluation metric to quantify color fidelity. Extensive experiments indicate that our method is more effective than other state-of-the-art image fusion methods, especially in color fidelity. The source code is available at https://github.com/GeoVectorMatrix/Dif-Fusion.

Geometric primitives that consist of a group of discrete points may be viewed as one kind of abstraction and representation of lidar data at the entity level. In recent years, many efforts from different scientific communities, such as photogrammetry, computer vision and computer graphics, have been made into geometric primitive detection, regularization, and in-depth applications. The most recent lidar-based surveys focus only on reconstruction, object segmentation, and recognition, as well as the data processing techniques based on a specific platform such as mobile LiDAR. However, in this paper, lidar point clouds are understood from a new perspective, i.e., geometric primitives embedded in versatile objects in physical world. We categorize geometric primitives into two classes: shape primitives, e.g., lines, surfaces, and volumetric shapes, as well as structure primitives, represented by skeletons and edges. Interpretations of geometric primitive from multiple disciplines try to convey the significance of geometric primitives, the latest processing techniques regarding geometric primitives, and their potential possibilities in the context of lidar point clouds. To this end, applications of these primitives are reviewed with an emphasis on object extraction and reconstruction to clearly show the significances of this paper. Next, we survey and compare methods for geometric primitive extraction and then survey primitive regularization methods that add real-world geometry constrains to detected primitives. Finally, we summarize the problems and challenges and describe possible future for primitive extraction methods that can be achieved globally optimal results efficiently, even with disorganized, uneven, noisy, incomplete, and large-scale lidar point clouds.

As a newly emerging advance in deep generative models, diffusion models have achieved state-of-the-art results in many fields, including computer vision, natural language processing, and molecule design. The remote sensing community has also noticed the powerful ability of diffusion models and quickly applied them to a variety of tasks for image processing. Given the rapid increase in research on diffusion models in the field of remote sensing, it is necessary to conduct a comprehensive review of existing diffusion model-based remote sensing papers, to help researchers recognize the potential of diffusion models and provide some directions for further exploration. Specifically, this paper first introduces the theoretical background of diffusion models, and then systematically reviews the applications of diffusion models in remote sensing, including image generation, enhancement, and interpretation. Finally, the limitations of existing remote sensing diffusion models and worthy research directions for further exploration are discussed and summarized.

Transformers have significantly advanced hyperspectral image (HSI) classification through their proficiency in modeling long sequences. However, the high dimensionality of HSI poses a particular challenge for Transformers due to their quadratic computational complexity. In natural language processing, state-space models such as Mamba hold great promise for handling long sequence tasks with significantly reduced computational overhead. However, the original Mamba lacks consideration for the spectral and spatial information inherent in HSI. Inspired by this, we propose the HyperMamba, a novel spectral-spatial adaptive Mamba for HSI classification. The core idea of HyperMamba involves adaptively scanning spatial neighborhood pixels and dynamically enhancing spectral bands for spectral scanning based on acquired spatial neighborhood information. Specifically, HyperMamba consists of two core modules: the Spatial Neighborhood Adaptive Scanning (SNAS) module and the Spectral Adaptive Enhancement Scanning (SAES) module. Initially, the SNAS module analyzes the spectral characteristics of classified pixels, adaptively selecting the optimal neighborhood for spatial scanning by balancing spatial neighborhood information and local spatial structure. Subsequently, the SAES module dynamically enhances the spectral features of classified pixels using neighborhood spectral information and conducts spectral scanning. Finally, the spectral features of the target pixels are fed into a single fully connected layer classifier, achieving high-precision HSI classification. Extensive experiments demonstrate the effectiveness of HyperMamba, surpassing state-of-the-art methods across three widely-used HSI datasets. The code will be available at https://github.com/chiangliu/HyperMamba.

Recently, there have been significant advancements in hyperspectral image (HSI) classification methods employing Transformer architectures. However, these methods, while extracting spectral-spatial features, may introduce irrelevant spatial information that interferes with HSI classification. To address this issue, this article proposes a spectral query spatial transformer (SQSFormer) framework. The proposed framework utilizes the center pixel (i.e., pixel to be classified) to adaptively query relevant spatial information from neighboring pixels, thereby preserving spectral features while reducing the introduction of irrelevant spatial information. Specifically, this article introduces a rotation-invariant position embedding (RIPE) module to integrate random central rotation (RR) and center relative position embedding (PE), mitigating the interference of absolute position and orientation information on spatial feature extraction. Moreover, a spectral-spatial center attention (SSCA) module is designed to enable the network to focus on the center pixel by adaptively extracting spatial features from neighboring pixels at multiple scales. The pivotal characteristic of the proposed framework achieves adaptive spectral-spatial information fusion using the Spectral Query Spatial paradigm, reducing the introduction of irrelevant information and effectively improving classification performance. Experimental results on multiple public datasets demonstrate that our framework outperforms previous state-of-the-art methods. For the sake of reproducibility, the source code of SQSFormer will be publicly available at https://github.com/chenning0115/SQSFormer .

Airborne light detection and ranging (LiDAR) data are widely applied in building reconstruction, with studies reporting success in typical buildings. However, the reconstruction of curved buildings remains an open research problem. To this end, we propose a new framework for curved building reconstruction via assembling and deforming geometric primitives. The input LiDAR point clouds are first converted into contours where individual buildings are identified. After recognizing geometric units (primitives) from building contours, we get initial models by matching the basic geometric primitives to these primitives. To polish assembly models, we employ a warping field for model refinements. Specifically, an embedded deformation (ED) graph is constructed via downsampling the initial model. Then, the point to model displacements is minimized by adjusting node parameters in the ED graph based on our objective function. The presented framework is validated on several highly curved buildings collected by various LiDAR in different cities. The experimental results, as well as accuracy comparison, demonstrate the advantage and effectiveness of our method. The new insight attributes to an efficient reconstruction manner. Moreover, we prove that the primitive-based framework significantly reduces the data storage to 10%-20% of classical mesh models.