Hyperspectral (HS) and multispectral (MS) image fusion aims at producing high-resolution HS (HRHS) images. However, the existing methods could not simultaneously consider the structures in both the spatial and spectral domains of the HS cube. In order to effectively preserve spatial-spectral structures in HRHS images, we propose a new low-resolution HS (LRHS) and high-resolution MS (HRMS) image fusion method based on spatial-spectral-graph-regularized low-rank tensor decomposition (SSGLRTD) in this paper. First, we reformulate the image fusion problem as a low-rank tensor decomposition model to utilize the low-rank property in the HS image. Then, two graphs are constructed in spatial and spectral domains, respectively. One of them is derived from the HRMS image for the spatial consistency, and the other is inferred from the LRHS image for spectral smoothness. Finally, the SSGLRTD fusion framework is established by combining all regularizers. With these two graphs, the spatial correlation and the spectral structure in the fused HRHS images are efficiently preserved. The experimental results on different datasets reveal that the proposed fusion method outperforms several existing fusion methods in terms of visual analysis and numerical comparison.

3D Gaussian Splatting (3DGS) has become an emerging tool for dynamic scene reconstruction. However, existing methods mainly focus on developing various strategies to extend static 3DGS into a time-variant representation, while overlooking the rich motion information implicitly carried by 2D observations, thus suffering from performance degradation and model redundancy. To address the above problem, we propose a novel motion-aware enhancement framework for dynamic scene reconstruction, which mines useful motion cues from optical flow to improve different paradigms of dynamic 3DGS. Specifically, we first step beyond the vanilla render-based cross-dimensional supervision that suffers from ambiguity and instability, and establish a more robust and effective dense correspondence between 3D Gaussian movements and pixel-level flows. Then a novel flow augmentation method is introduced with additional insights into uncertainty and loss collaboration. Furthermore, for the prevalent deformation-based paradigm that presents a harder optimization problem, a transient-aware deformation auxiliary module is proposed. We conduct extensive experiments on both multi-view and monocular scenes to verify the merits of our work. Compared with the baselines, our method shows significant superiority in both rendering quality and efficiency. The code will be publicly available at https://github.com/jasongzy/MAGS.

Internet of Things (IoT) networks are often subject to many malicious attacks in untrusted environments, and automatic modulation classification (AMC) is an effective way to combat IoT physical-layer threats. However, most existing AMC methods assume sufficient labeled signals and invariant signal distribution, which is often impossible in untrusted environments. In this article, a new meta-learning method is proposed for a few-shot AMC with distribution bias. First, a multi-frequency octave ResNet (MFOR) is constructed to learn coarse (low-frequency) and fine (high-frequency) features, which can efficiently identify the modulation type of the signal while saving computational resources. Second, a large number of classification-related meta-tasks are established for training MFOR to explore general knowledge in signal classification, and then transfer it to the AMC. Different with deep neural networks (DNNs) that learn a mapping by multiple instances, the MFOR with meta-learning (denoted as M-MFOR) can improve the generalization ability of new AMC tasks with very few instances and distribution bias. Furthermore, we find that the distribution bias between data can be reduced by adjusting the normalized distribution and propose a class-related mixup. Extensive experiments are taken on several datasets to investigate the effectiveness of M-MFOR. The results show its feasibility and superiority over existing methods.

In embodied vision, Instance ImageGoal Navigation (IIN) requires an agent to locate a specific object depicted in a goal image within an unexplored environment. The primary challenge of IIN arises from the need to recognize the target object across varying viewpoints while ignoring potential distractors. Existing map-based navigation methods typically use Bird’s Eye View (BEV) maps, which lack detailed texture representation of a scene. Consequently, while BEV maps are effective for semantic-level visual navigation, they are struggling for instance-level tasks. To this end, we propose a new framework for IIN, Gaussian Splatting for Visual Navigation (GaussNav), which constructs a novel map representation based on 3D Gaussian Splatting (3DGS). The GaussNav framework enables the agent to memorize both the geometry and semantic information of the scene, as well as retain the textural features of objects. By matching renderings of similar objects with the target, the agent can accurately identify, ground, and navigate to the specified object. Our GaussNav framework demonstrates a significant performance improvement, with Success weighted by Path Length (SPL) increasing from 0.347 to 0.578 on the challenging Habitat-Matterport 3D (HM3D) dataset.

Finding accurate injection components is the key issue in pan-sharpening methods. In this paper, a low-rank pan-sharpening (LRP) model is developed from a new perspective of offset learning. Two offsets are defined to represent the spatial and spectral differences between low-resolution multispectral and high-resolution multispectral (HRMS) images, respectively. In order to reduce spatial and spectral distortions, spatial equalization and spectral proportion constraints are designed and cast on the offsets, to develop a spatial and spectral constrained stable low-rank decomposition algorithm via augmented Lagrange multiplier. By fine modeling and heuristic learning, our method can simultaneously reduce spatial and spectral distortions in the fused HRMS images. Moreover, our method can efficiently deal with noises and outliers in source images, for exploring low-rank and sparse characteristics of data. Extensive experiments are taken on several image data sets, and the results demonstrate the efficiency of the proposed LRP.

Cross-modal retrieval aims to identify relevant data across different modalities. In this work, we are dedicated to cross-modal retrieval between images and text sentences, which is formulated into similarity measurement for each image-text pair. To this end, we propose a Cross-modal Relation Guided Network (CRGN) to embed image and text into a latent feature space. The CRGN model uses GRU to extract text feature and ResNet model to learn the globally guided image feature. Based on the global feature guiding and sentence generation learning, the relation between image regions can be modeled. The final image embedding is generated by a relation embedding module with an attention mechanism. With the image embeddings and text embeddings, we conduct cross-modal retrieval based on the cosine similarity. The learned embedding space well captures the inherent relevance between image and text. We evaluate our approach with extensive experiments on two public benchmark datasets, i.e., MS-COCO and Flickr30K. Experimental results demonstrate that our approach achieves better or comparable performance with the state-of-the-art methods with notable efficiency.