Existing fusion methods empirically design elaborate fusion losses to retain the specific features from source images. Since image fusion has no ground truth, the hand-crafted losses may not make the fused images cover all the vital features, and then affect the performance of the high-level tasks. Here, there are two main challenges: domain discrepancy among source images and semantic mismatch at different-level tasks. This paper proposes an infrared and visible image fusion via cross reconstruction learning, which doesn't using any hand-crafted fusion losses, but prompts the network to adaptively fuse complementary information of source images. Firstly, we design a cross reconstruction learning model that decouples the fusion features to reconstruct another-modality source image. Thus, the fusion network is forced to learn the domain-adaptive representations of two modal features, which enables their domain alignment in a latent space. Secondly, we propose a dynamic interactive fusion strategy that builds a correlation matrix between fusion features and object semantic features to overcome the semantic mismatch. Further, we enhance the strong correlation features and suppress the weak correlation features to improve the interactive ability. Extensive experiments on three datasets demonstrate the superior fusion performance compared to the state-of-the-art methods, concurrently facilitating the segmentation accuracy.

With the development of the Industrial Internet of Things (IoT), the work of large-scale data collection makes spatiotemporal crowdsensing (SC) play an important role. Mobile devices equipped with sensors could act as workers to collect and process data for uploading. In the task allocation process, a fully static allocation fails to meet the needs of realistic conditions, while a completely dynamic allocation fails to achieve the desired results. Therefore, we assume a task-scheduled execution scenario that combines the above two conditions. In the pre-allocation process, an original time location constraints (ORTA) allocation algorithm is first proposed. Then it is optimized (OPTA) to fully utilize the remaining time of the workers and increase the matched number. In addition, the design of the incentive mechanism is an effective means to improve the task completion rate of the platform. To efficiently utilize the limited platform budget in the long run, a Q-learning-based algorithm is proposed to identify target inspire tasks and subsequently increase their reward to attract workers’ participation. Finally, comparison experiments are conducted on real datasets to verify the effectiveness of our algorithm. Furthermore, the experiments on a Raspberry Pi local terminal are conducted under a satellite-based environment.

The emerging research line of cross-modal learning focuses on the issue of transferring feature representation manner learned from limited multimodal data with labelings to the testing phase with partial modalities. This is essentially common and practical in the remote sensing community when only modal-incomplete data are in users’ hands due to inevitable imaging or access restrictions under large-scale observation scenarios. However, most of the existing cross-modal learning methods have been designed with exclusive reliance on labeling, which can be either limited or noisy due to their costly production. To address this issue, we explore in this paper the possibility to learn cross-modal feature representation in an unsupervised fashion. By integrating the multimodal data into a fully recombined matrix form, we propose 1) the use of common subspace representation as the regression target instead of conventionally adopted binary labels, and 2) the orthogonality and manifold alignment regularization terms to shrink the solution space whilst preserving the pairwise manifold correlations. Through this manner, the modality-specific and mutual latent representations in this common subspace as well as their corresponding projections can be learned simultaneously and their optimums can be efficiently reached through a nearly one-step computation with the help of Eigen decomposition. Finally, we show the superiority of our method through extensive image classification experiments on three multimodal datasets with four remotely sensed modalities involved (i.e., hyperspectral, multispectral, synthetic aperture radar, and light detection and ranging data). The code and dataset will be made freely available at https://github.com/jingyao16/UCSL after a possible publication to encourage the reproduction of our method and further use.

Nowadays, the proliferation of small satellites brings the skyrocketing rise in space data, especially the shift to on-orbit computing needs. On one hand, with the increasing volume of data generation, like high-resolution remote sensing images, on-orbit computing produces near real-time onboard solutions and quick responses. However, constrained by the limited size and energy supply of satellites, achieving energy-efficient on-orbit computing remains a crucial challenge. In this article, an on-orbit remote sensing image processing complex task scheduling model facing heterogeneous multiprocessor system (HMPS) is proposed. First, aiming at accelerating image processing, we establish a novel parallel task execution model using directed acyclic graph (DAG) to universally describe typical missions, i.e., cloud detection, geometric correction, and image classification. Subsequently, a mathematical task scheduling formulation is defined to calculate the makespan, and total energy consumption (TEC) required when executing DAG on HMPS. Second, a new Pareto-based iterated greedy optimizer (PIGO) is devised to complete the energy- and time-efficient task execution and resource allocation on HMPS through confined inserting mutation, destruction-reconstruction, and local search. Finally, we build an emulated on-orbit HMPS to conduct experiments. The results show that, in comparison with the scheme without model scheduling, the most savings of around 51% and 54% in makespan and TEC, respectively, are achieved by the proposed model. Moreover, the HMPS configured with our methodology can obtain 2.2× improvement in energy efficiency and process up to 2.56×10 ⁵ pixels per unit of power (W) and time (s).

Multimodal image fusion has recently garnered increasing interest in the field of remote sensing. By leveraging the complementary information in different modalities, the fused results may be more favorable in characterizing objects of interest, thereby increasing the chance of a more comprehensive and accurate perception of the scene. Unfortunately, most existing fusion methods tend to extract modality-specific features independently without considering inter-modal alignment and complementarity, leading to a suboptimal fusion process. To address this issue, we propose a novel interactive guided generative adversarial network, named IG-GAN, for the task of multi-modal image fusion. IG-GAN comprises guided dual streams tailored for enhanced learning of details and content, as well as cross-modal consistency. Specifically, a details-guided interactive running-in module and a content-guided interactive running-in module are developed, with the stronger modality serving as guidance for detail richness or content integrity, and the weaker one assisting. To fully integrate multi-granularity features from dual-modality, a hierarchical fusion and reconstruction branch is established. Specifically, a shallow interactive fusion module followed by a multi-level interactive fusion module is designed to aggregate multi-level local and long-range features. Concerning feature decoding and fused image generation, a high-level interactive fusion and reconstruction module is further developed. Additionally, to empower the fusion network to generate fused images with complete content, sharp edges, and high fidelity without supervision, a loss function facilitating the mutual game between the generator and two discriminators is also formulated. Comparative experiments with fourteen state-of-the-art methods are conducted on three datasets. Qualitative and quantitative results indicate that IG-GAN exhibits obvious superiority in terms of both visual effect and quantitative metrics. Moreover, experiments on two RGB-IR object detection datasets are also conducted, which demonstrate that IG-GAN can enhance the accuracy of object detection by integrating complementary information from different modalities.The code will be available at https://github.com/flower6top.