In recent years, deep learning-based hyperspectral unmixing techniques have garnered increasing attention and made significant advancements. However, relying solely on the use of CNN or Transformer approaches is insufficient for effectively capturing both global and fine-grained information, thereby compromising the accuracy of unmixing tasks. In order to fully harness the information contained within HSIs, this study explores a dual-stream collaborative network, referred to as TCCU-Net. It end-to-end learns information in four dimensions: spectral, spatial, global, and local, to achieve more effective unmixing. The network comprises two core encoders: one is a Transformer encoder, which includes squeeze-launch modules, DSSCR-VIT modules, and stripe pooling modules, while the other one is a CNN encoder, which is composed of 2D pyramid convolutions and 3D pyramid convolutions. By fusing the outputs of these two encoders, the semantic gap between the encoder and decoder is bridged, resulting in improved feature mapping and unmixing outcomes. This study extensively evaluates TCCU-Net and seven hyperspectral unmixing methods on four datasets (Samson, Apex, Jasper Ridge and Synthetic dataset). The experimental results firmly demonstrate that the proposed approach surpasses others in terms of accuracy, holding the potential to effectively address hyperspectral unmixing tasks.
30 October 2024 in IEEE Geoscience and Remote Sensing Letters
Integration of complementary information from different modalities and efficient computation is crucial in remote sensing image classification applications. Convolutional neural networks (CNN), recurrent neural networks (RNN), and transformers have achieved great success in multimodal remote sensing classification, but their practical utility is constrained by the challenges of parallelization and the inefficiencies associated with overemphasis on computational complexity. To address these issues, we propose S 2 CrossMamba based on a state-space model (SSM), an efficient multimodal fusion network tailored for hyperspectral image (HSI) data. First, the Cross-SSM is designed to dynamically update the multimodal state information to fuse different modalities with all parameters learned multimodal input-dependent, yielding a centralized response. Secondly, the inverted bottleneck Cross-Mamba (IBCM) is designed based on Cross-SSM to increase the feature richness by mixing the multimodal features through the inverted bottleneck structure, and the mixed features are efficiently operated through Cross-SSM. To further enhance the HSI spatial and spectral feature extraction capability, dual-branch spectral Mamba and spatial Mamba are designed for feature fusion and learning based on spatial and spectral attention and IBCM structure. Our network achieves higher performance with a smaller number of parameters. The codes will be released at https://github.com/HyperSystemAndImageProc/S2CrossMamba.
In recent years, convolutional neural networks (CNN)‐based automatic segmentation of medical images has become one of the hot topics in clinical disease diagnosis. It is still a challenging task to improve the segmentation accuracy of the network model with the large variation of pathological regions in different patients and the fuzzy boundary of pathological regions. A Dual‐path Multi‐scale Attention Guided network (DMAGNet) for medical image segmentation is proposed in this paper. First, the Dual‐path Multi‐scale Attention Fusion Module (DMAF) is proposed as a novel skip connection strategy, which is applied to encode semantic dependencies between high‐level and low‐level channels. Second, the Multi‐scale Normalized Channel Attention Module (MNCA) based on the atrous convolution, normalization channel attention mechanism, and the Depthwise Separable Convolutions (DSConv) is developed to strengthen dependencies between channels. Finally, the encoder–decoder backbone employs the DSConv, as well as the pretrained Resnet34 block is combined in the encoder part to further improve the backbone network performance. Comprehensive experiments on brain, lung, and liver segmentation tasks show that the proposed DMAGNet outperforms the original U‐Net method and other advanced methods.
In this paper, a regular dodecahedron solar simulator structure with five spot light sources is presented to provide uniform flux. To analyze light distribution without a dodecahedron cavity, a cubic cosine relationship between deflection angle and test flat plane is proposed, and results show that nonuniformity reaches 5%. By introducing a regular dodecahedron cavity, nonuniformity can be reduced to 2% in a circular area with an 85% inradius, according to the Monte Carlo ray tracing method. Based on the structure, a prototype simulator is fabricated by pentagons with 4 cm inradius and with five high-power light-emitting diodes (LEDs). Across the inscribed circular aperture of the bottom light output pentagon, nonuniformity is 3.1%; while 2% nonuniformity is reached in a 3.4-cm 2 circular aperture whose area occupies 72% of the inscribed circle.
14 October 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Methods for joint classification of hyperspectral images (HSI) with high dimensionality and spectral correlation and other sensor data (e.g., optical, infrared, radar, etc.) are important directions in the field of remote sensing. To better learn the feature representation of HSI, the unsupervised global modeling property of Diffusion is utilized to mine the potential features of HSI to obtain Diffusion features as input data. In addition, to fuse HSI features, HSI Diffusion features (DHSI), and other data features. A three-input DECT network based on CNN and Transformer is proposed for feature extraction and fusion. Firstly, the primary features are extracted by hierarchical CNN after pre-modal fusion. Second, considering the high dimensionality of HSI, Spectral Pooling Attention Interaction (SPAI) is designed for feature extraction and aggregation of information from different attentions. Finally, the Inverted Bottleneck Convolutional Transformer (IBCT) is designed to aggregate multi-source information to enhance feature reuse and aggregate local and contextual information. It is shown on three publicly available datasets that DECT outperforms current state-of-the-art methods