Transformer has been extensively explored for hyperspectral image classification. However, transformer poses challenges in terms of speed and memory usage because of its quadratic computational complexity. Recently the Mamba model has emerged as a promising approach, which has strong long-distance modeling capabilities while maintaining a linear computational complexity. However, representing the hyperspectral image is challenging for the Mamba due to the requirement for an integrated spatial and spectral understanding. To remedy these drawbacks, we propose a novel hyperspectral image classification model based on a Mamba model, named MambaHSI, which can simultaneously model long-range interaction of the whole image and integrate spatial and spectral information in an adaptive manner. Specifically, we design a spatial Mamba block to model the long-range interaction of the whole image at the pixel-level. Then we propose a spectral Mamba block to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features. Finally, we propose a spatial-spectral fusion module to adaptively integrate spatial and spectral features of a hyperspectral image. To our best knowledge, this is the first image-level hyperspectral image classification model based on the Mamba. We conduct extensive experiments on four diverse hyperspectral image datasets. The results demonstrate the effectiveness and superiority of the proposed model for hyperspectral image classification. This reveals the great potential of Mamba to be the next-generation backbone for hyperspectral image models. Codes are available at https://github.com/li-yapeng/MambaHSI.

Domain adaptation aims to exploit the supervision knowledge in a source domain for learning prediction models in a target domain. In this article, we propose a novel representation learning-based domain adaptation method, i.e., neural embedding matching (NEM) method, to transfer information from the source domain to the target domain where labeled data is scarce. The proposed approach induces an intermediate common representation space for both domains with a neural network model while matching the embedding of data from the two domains in this common representation space. The embedding matching is based on the fundamental assumptions that a cross-domain pair of instances will be close to each other in the embedding space if they belong to the same class category, and the local geometry property of the data can be maintained in the embedding space. The assumptions are encoded via objectives of metric learning and graph embedding techniques to regularize and learn the semisupervised neural embedding model. We also provide a generalization bound analysis for the proposed domain adaptation method. Meanwhile, a progressive learning strategy is proposed and it improves the generalization ability of the neural network gradually. Experiments are conducted on a number of benchmark data sets and the results demonstrate that the proposed method outperforms several state-of-the-art domain adaptation methods and the progressive learning strategy is promising.

Constructing a 3D scene capable of accommodating open-ended language queries, is a pivotal pursuit in the domain of robotics, which facilitates robots in executing object manipulations based on human language directives. To achieve this, some research efforts have been dedicated to the development of language-embedded implicit fields. However, implicit fields (e.g. NeRF) encounter limitations due to the necessity of taking images from a larger number of viewpoints for reconstruction, coupled with their inherent inefficiencies in inference. Furthermore, these methods directly distill patch-level 2D features, leading to ambiguous segmentation boundaries. Thus, we present the GaussianGrasper , which uses 3D Gaussian Splatting (3DGS) to explicitly represent the scene as a set of Gaussian primitives and is capable of real-time rendering. Our approach takes RGB-D images from limited viewpoints as input and uses an Efficient Feature Distillation (EFD) module that employs contrastive learning to efficiently distill 2D language embeddings and constraint consistency of feature embeddings. With the reconstructed geometry of the Gaussian field, our method enables the pre-trained grasping model to generate collision-free grasp pose candidates. Furthermore, we propose a normal-guided grasp module to select the best grasp pose. Through comprehensive real-world experiments, we demonstrate that GaussianGrasper enables robots to accurately locate and grasp objects according to language instructions, providing a new solution for language-guided grasping tasks.

Hyperspectral remote sensing image classification has drawn a great attention in recent years due to the development of remote sensing technology. To build a high confident classifier, the large number of labeled data is very important, e.g., the success of deep learning technique. Indeed, the acquisition of labeled data is usually very expensive, especially for the remote sensing images, which usually needs to survey outside. To address this problem, in this letter, we propose a domain adaptation method by learning the manifold embedding and matching the discriminative distribution in source domain with neural networks for hyperspectral image classification. Specifically, we use the discriminative information of source image to train the classifier for the source and target images. To make the classifier can work well on both domains, we minimize the distribution shift between the two domains in an embedding space with prior class distribution in the source domain. Meanwhile, to avoid the distortion mapping of the target domain in the embedding space, we try to keep the manifold relation of the samples in the embedding space. Then, we learn the embedding on source domain and target domain by minimizing the three criteria simultaneously based on a neural network. The experimental results on two hyperspectral remote sensing images have shown that our proposed method can outperform several baseline methods.

Due to the fact that it is expensive and time-consuming to annotate a large amount of data, the available labeled data to train a deep neural network is usually scarce, resulting in the poor performance of the deep classification network in most cases. In this paper, we propose a novel active learning method to train a competitive deep classification neural network by labeling a limited amount of diverse images. The proposed method advances most of the active learning methods in two aspects. First, active learning is enhanced with a co-auxiliary learning strategy. We use an auxiliary network to provide diverse pseudo-labels for the primary network. The auxiliary network is also adopted to remove a part of redundancy information from the candidate pool of active learning when image predictions of the primary network and the auxiliary network are the same. Meanwhile, the primary network can also provide pseudo-labels to improve the performance of the auxiliary network. Second, we further remove the redundancy within the query samples with multi-level diversity selection. The multi-level diversity strategy not only considers the feature-level diversity but also the class-level diversity with the predictions of the primary network, and it can select both representative and uncertain samples effectively and efficiently from the large-scale data. The extensive experiments on four large-scale datasets show that the proposed method outperforms the state-of-the-art methods.