In recent years, deep learning-based hyperspectral unmixing techniques have garnered increasing attention and made significant advancements. However, relying solely on the use of CNN or Transformer approaches is insufficient for effectively capturing both global and fine-grained information, thereby compromising the accuracy of unmixing tasks. In order to fully harness the information contained within HSIs, this study explores a dual-stream collaborative network, referred to as TCCU-Net. It end-to-end learns information in four dimensions: spectral, spatial, global, and local, to achieve more effective unmixing. The network comprises two core encoders: one is a Transformer encoder, which includes squeeze-launch modules, DSSCR-VIT modules, and stripe pooling modules, while the other one is a CNN encoder, which is composed of 2D pyramid convolutions and 3D pyramid convolutions. By fusing the outputs of these two encoders, the semantic gap between the encoder and decoder is bridged, resulting in improved feature mapping and unmixing outcomes. This study extensively evaluates TCCU-Net and seven hyperspectral unmixing methods on four datasets (Samson, Apex, Jasper Ridge and Synthetic dataset). The experimental results firmly demonstrate that the proposed approach surpasses others in terms of accuracy, holding the potential to effectively address hyperspectral unmixing tasks.

Integration of complementary information from different modalities and efficient computation is crucial in remote sensing image classification applications. Convolutional neural networks (CNN), recurrent neural networks (RNN), and transformers have achieved great success in multimodal remote sensing classification, but their practical utility is constrained by the challenges of parallelization and the inefficiencies associated with overemphasis on computational complexity. To address these issues, we propose S ² CrossMamba based on a state-space model (SSM), an efficient multimodal fusion network tailored for hyperspectral image (HSI) data. First, the Cross-SSM is designed to dynamically update the multimodal state information to fuse different modalities with all parameters learned multimodal input-dependent, yielding a centralized response. Secondly, the inverted bottleneck Cross-Mamba (IBCM) is designed based on Cross-SSM to increase the feature richness by mixing the multimodal features through the inverted bottleneck structure, and the mixed features are efficiently operated through Cross-SSM. To further enhance the HSI spatial and spectral feature extraction capability, dual-branch spectral Mamba and spatial Mamba are designed for feature fusion and learning based on spatial and spectral attention and IBCM structure. Our network achieves higher performance with a smaller number of parameters. The codes will be released at https://github.com/HyperSystemAndImageProc/S2CrossMamba.

Due to the extremely high manufacturing standards, the integration of quasi-omnidirectional photodetectors and synaptic devices within a single device remains a long-standing challenge. In this work, we have designed a graphene/(Al,Ga)N nanowire heterojunction, demonstrating the monolithic integration of self-driven 360° photodetectors and artificial synapses in a dual-mode transparent device successfully. By manipulating the carrier transport dynamics through controlling the bias voltage, the degree of oxygen vacancy ionization can be precisely regulated, ultimately realizing the monolithic dual-mode device. At 0 V bias, the device functions as a fast-response self-driven photodetector with stable optical communication capabilities, achieving 360° quasi-omnidirectional photodetection. Upon applying a bias voltage, the operating mode switches to a synaptic device, which successfully simulates brain-like paired-pulse facilitation, short-/long-term plasticity processes, and learning/forgetting behaviors. The device demonstrates an exceptionally high UV/visible rejection ratio of 1.29 × 10⁴, coupled with an ultra-low dark current of less than 1 pA. Furthermore, this device has a low power consumption of 2.5 × 10⁻¹⁴ J per synaptic event, indicating an energy efficiency comparable to synaptic processes in the human brain. Moreover, nonlinear photoconductivity lets the device become a neuromorphic sensor for preprocessing images, enhancing recognition accuracy. Importantly, by leveraging the long-memory characteristic of the devices in open-circuit voltage mode, the devices have been successfully applied to guide humanoid robots in performing direction distinguishing and motion learning. This work provides new insights into the integrated manufacturing of multifunctional monolithic devices and foresees their immense potential in upcoming advanced, low-power neuromorphic computing systems.

Methods for joint classification of hyperspectral images (HSI) with high dimensionality and spectral correlation and other sensor data (e.g., optical, infrared, radar, etc.) are important directions in the field of remote sensing. To better learn the feature representation of HSI, the unsupervised global modeling property of Diffusion is utilized to mine the potential features of HSI to obtain Diffusion features as input data. In addition, to fuse HSI features, HSI Diffusion features (DHSI), and other data features. A three-input DECT network based on CNN and Transformer is proposed for feature extraction and fusion. Firstly, the primary features are extracted by hierarchical CNN after pre-modal fusion. Second, considering the high dimensionality of HSI, Spectral Pooling Attention Interaction (SPAI) is designed for feature extraction and aggregation of information from different attentions. Finally, the Inverted Bottleneck Convolutional Transformer (IBCT) is designed to aggregate multi-source information to enhance feature reuse and aggregate local and contextual information. It is shown on three publicly available datasets that DECT outperforms current state-of-the-art methods

Hyperspectral image (HSI) classification has become a popular research topic in recent years, and transformer-based networks have demonstrated superior performance by analyzing global semantic features. However, using transformers for pixel-level HSI classification has two limitations: ineffective capture of spatial-spectral correlations and inadequate exploitation of local features. To address these challenges, we propose a Dual-Dimension Self-Attention (D ² SA) mechanism that fully exploits HSI's high spectral-spatial correlation by using two separate branches to model the global dependence of features from the spectral and spatial dimensions. Additionally, we develop a multi-layer residual convolution module that extracts local features and introduces shallow-deep feature interactions to obtain more discriminative representations. Based on these components, we propose a Dual Dimension Spectral-Spatial Bottleneck Transformer (D ² S ² BoT) framework for HSI classification that simultaneously models the local interactions and global dependencies of HSI pixels to achieve high-precision classification. By virtue of the D ² SA mechanism, the introduced D ² S ² BOT framework can produce competitive classification results with a limited number of training samples on three well-known datasets, which we hope will provide a strong baseline for future research on transformers in the field of HSI. Codes are released at: https://github.com/HyperSystemAndImageProc/D2S2BoT .