The fusion of infrared and visible images can utilize the indication characteristics and the textural details of source images to realize the all-weather detection. The deep learning (DL) based fusion solutions can reduce the computational cost and complexity compared with traditional methods since there is no need to design complex feature extraction methods and fusion rules. There are no standard reference images and the publicly available infrared and visible image pairs are scarce. Most supervised DL-based solutions have to take pre-training on other labeled large datasets which may not behave well when testing. The few unsupervised fusion methods can hardly obtain ideal images with good visual impression. In this paper, an infrared and visible image fusion method based on unsupervised convolutional neural network is proposed. When designing the network structure, densely connected convolutional network (DenseNet) is used as the sub-network for feature extraction and reconstruction to ensure that more information of source images can be retained in the fusion images. As to loss function, the perceptual loss is creatively introduced and combined with the structure similarity loss to constrain the updating of weight parameters during the back propagation. The perceptual loss designed helps to improve the visual information fidelity (VIF) of the fusion image effectively. Experimental results show that this method can obtain fusion images with prominent targets and obvious details. Compared with other 7 traditional and deep learning methods, the fusion results of this method are better on objective evaluation and visual observation when taken together.
Change detection (CD) in high-resolution remote sensing imagery remains challenging due to the complex nature of objects and varying spectral characteristics across different times and locations. Convolutional neural networks (CNNs) have shown promising performance in CD tasks by extracting meaningful semantic features. However, traditional 2D-CNNs may struggle to accurately integrate deep features from multi-temporal images, limiting their ability to improve CD accuracy. This study proposes a Multi-level Feature Cross-Fusion (MFCF) network with 3D-CNNs for remote sensing image change detection. The network aims to effectively extract and fuse deep features from multi-temporal images to identify surface changes. To bridge the semantic gap between high-level and low-level features, a MFCF module is introduced. A channel attention mechanism (CAM) is also integrated to enhance model performance, interpretability, and generalization capabilities. The proposed methodology is validated on the LEVIR construction dataset (LEVIR-CD). The experimental results demonstrate superior performance compared to the current state-of-the-art in evaluation metrics including recall, F1 score, and IOU. The MFCF network, which combines 3D-CNNs and a CAM, effectively utilizes multi-temporal information and deep feature fusion, resulting in precise and reliable change detection in remote sensing imagery. This study significantly contributes to the advancement of change detection methods, facilitating more efficient management and decision making across various domains such as urban planning, natural resource management, and environmental monitoring.
In applications such as satellite remote sensing and aerial photography, imaging equipment must capture brightness information of different ground scenes within a restricted dynamic range. Due to camera sensor limitations, captured images can represent only a portion of such information, which results in lower resolution and lower dynamic range compared with real scenes. Image super resolution (SR) and multiple-exposure image fusion (MEF) are commonly employed technologies to address these issues. Nonetheless, these two problems are often researched in separate directions. In this paper, we propose MEFSR-GAN: an end-to-end framework based on generative adversarial networks that simultaneously combines super-resolution and multiple-exposure fusion. MEFSR-GAN includes a generator and two discriminators. The generator network consists of two parallel sub-networks for under-exposure and over-exposure, each containing a feature extraction block (FEB), a super-resolution block (SRB), and several multiple-exposure feedback blocks (MEFBs). It processes low-resolution under- and over-exposed images to produce high-resolution high dynamic range (HDR) images. These images are evaluated by two discriminator networks, driving the generator to generate realistic high-resolution HDR outputs through multi-goal training. Extensive qualitative and quantitative experiments were conducted on the SICE dataset, yielding a PSNR of 24.821 and an SSIM of 0.896 for 2× upscaling. These results demonstrate that MEFSR-GAN outperforms existing methods in terms of both visual effects and objective evaluation metrics, thereby establishing itself as a state-of-the-art technology.
Accurate quality assessment of solar Extreme Ultraviolet (EUV) remote sensing imagery is critical for data reliability in space science and weather forecasting. This study introduces a hybrid framework that fuses deep semantic features from a HyperNet-based model with 22 handcrafted physical and statistical quality indicators to create a robust 24-dimensional feature vector. We used a dataset of top-quality images, i.e., quality class “Excellent”, and generated a dataset of 47,950 degraded, lower-quality images by simulating seven types of degradation including defocus, blur and noise. Experimental results show that an XGBoost classifier, when trained on these fused features, achieved superior performance with 97.91% accuracy and an AUC of 0.9992. This approach demonstrates that combining deep and handcrafted features significantly enhances the classification’s robustness and offers a scalable solution for automated quality control in solar EUV observation pipelines.
To meet the requirements of quantitative elemental analysis in the ultraviolet (UV) spectrum, a UV single-photon imaging system was developed, integrating a digital micromirror device (DMD) and a single photon-counting imaging detector, enabling high sensitivity, high resolution, and a wide dynamic range. However, intrinsic geometric distortion poses a significant challenge to accurate spectral calibration. A hybrid correction framework is proposed, cascading polynomial coarse correction with multilayer perceptron (MLP) fine regression, improving calibration accuracy. The method utilizes a full-field dot-array mask projected by the DMD to acquire distortion-reference image pairs. The polynomial model rapidly captures the dominant high-order distortion, while a lightweight MLP performs non-parametric fine regression of residual displacements, achieving a mean error of 0.84 pixels. This approach reduces the root mean square (RMS) error to 1.01 pixels, outperforming traditional direct linear transformation (5.35 pixels) and pure polynomial models (1.33 pixels), while the nonlinearity index decreases from 0.35° to 0.05°. In addition, the method demonstrates stable performance across multi-scale checkerboard patterns ranging from 128 to 280 pixels, with RMS errors remaining around the 1-pixel level. These results validate the high-precision distortion suppression and robust cross-scale performance of the proposed framework. By leveraging DMD-generated patterns for self-calibration, this method eliminates the need for external targets, offering a scalable solution for high-end spectrometer calibration.