The development of deep learning has brought new prospects into the field of hyperspectral classification, and the classification ability of this method for the classification of hyperspectral images (HSIs) has been continuously improved. However, there are still some problems that must be solved; for example, the spectral–spatial features of HSIs are not effectively extracted, the labeled samples in the data set are limited, and the number of samples in different categories is imbalanced. To facilitate the progress of hyperspectral classification, a spectral–spatial fractal residual convolutional neural network with data balance augmentation is proposed here. In this network, a data balance augmentation approach that can solve the problems of limited labeled data and imbalanced categories is proposed. In addition, the spectral–spatial residual module is proposed to learn the spectral–spatial information and alleviate the problem of model degradation effectively. In addition, the spectral–spatial focal structure, which can guarantee the integrity of the information, is introduced. Moreover, the spectral–spatial dimensional transformation module, which can reduce the size and number of hyperspectral feature maps without losing the fine features, is presented. In particular, the proposed network has a strong ability to classify the categories that have a small number of samples and reaches the state-of-the-art level for three benchmark data sets.

Infrared and visible image fusion can obtain combined images with salient hidden objectives and abundant visible details simultaneously. In this paper, we propose a novel method for infrared and visible image fusion with a deep learning framework based on a generative adversarial network (GAN) and a residual network (ResNet). The fusion is accomplished with an adversarial game and directed by the unique loss functions. The generator with residual blocks and skip connections can extract deep features of source image pairs and generate an elementary fused image with infrared thermal radiation information and visible texture information, and more details in visible images are added to the final images through the discriminator. It is unnecessary to design the activity level measurements and fusion rules manually, which are now implemented automatically. Also, there are no complicated multi-scale transforms in this method, so the computational cost and complexity can be reduced. Experiment results demonstrate that the proposed method eventually gets desirable images, achieving better performance in objective assessment and visual quality compared with nine representative infrared and visible image fusion methods.

Image super-resolution (SR) technique can improve the spatial resolution of images without upgrading the imaging system. As a result, SR promotes the development of high resolution (HR) remote sensing image applications. Many remote sensing image SR algorithms based on deep learning have been proposed recently, which can effectively improve the spatial resolution under the constraints of HR images. However, images acquired by remote sensing imaging devices typically have lower resolution. Hence, an insufficient number of HR remote sensing images are available for training deep neural networks. In view of this problem, we propose an unsupervised SR method that does not require HR remote sensing images. The proposed method introduces a generative adversarial network (GAN) that obtains SR images through the generator; then, the SR images are downsampled to train the discriminator with low resolution (LR) images. Our method outperformed several methods in terms of the quality of the obtained SR images as measured by 6 evaluation metrics, which proves the satisfactory performance of the proposed unsupervised method for improving the spatial resolution of remote sensing images.

Text-to-Image generative models have shown a remarkable ability to produce high-quality images. However, existing methods still face difficulties in exemplar-guided image editing without destroying the given objects’ identity in the exemplar image. To address this problem, we propose a new framework called Paste and Harmonize via Denoising, which leverages pre-trained diffusion models to facilitate the text-driven transfer of objects from an exemplar image to the edited image while preserving their appearance and characteristics. The framework consists of two main steps: paste and harmonize via denoising. In the paste step, an off-the-shelf text-driven model is utilized to localize the objects in the exemplar image. The editing task is naturally transformed into an image harmonization task by pasting the object patches into the edited image. In the harmonize via denoising step, we introduce an image harmonization module based on pre-trained diffusion models to blend the inserted object with the target image, producing a coherent and realistic image without compromising synthesis quality and preserving the text-driven style transfer editing ability. In the experiments, the qualitative comparisons with baselines demonstrate that our method achieves impressive performance in exemplar-based image editing on both training and in-the-wild images with high fidelity. More qualitative and quantitative results can be found at our website.

Recently, the classification of hyperspectral images has made great process. Especially, the classification methods based on three-dimensional convolutional neural network have remarkable performance due to the uniqueness of hyperspectral images. However, the hyperspectral classification still faces great challenges due to a series of problems such as the insufficient extraction of spectral-spatial features, the lack of labeled samples, the large amount of noise, the tendency of overfitting and so on. Therefore, SSDANet is proposed to solve the above problems and promote the further development of hyperspectral classification technology based on deep learning. SSDANet is a spectral-spatial three-dimensional convolutional neural network with a deep and wide structure that can significantly improve classification performance. In SSDANet, the spectral-spatial dense connectivity is put forward to protect the integrity of information. It is made up of the spectral branch and the spatial branch, which can learn and reuse the spectral-spatial features. Besides, the spectral-spatial attention mechanism is proposed to adapt the special structure of hyperspectral images. It can excite important spectral-spatial information and suppress unimportant spectral-spatial information. In addition, a series of optimization methods including data augmentation, batch normalization, dropout, exponential decay learning rate, and L2 regularization are adopted to alleviate the problem of overfitting and improve the classification results. To verify the performance of SSDANet, experiments were implemented on two widely used datasets—Pavia University and Indian Pines. Under the condition of limited labeled samples, the classification evaluation indexes of OA, AA, and Kappa on the two datasets all exceeded 99%, reaching state-of-the-art performance.

The fusion of infrared and visible images can utilize the indication characteristics and the textural details of source images to realize the all-weather detection. The deep learning (DL) based fusion solutions can reduce the computational cost and complexity compared with traditional methods since there is no need to design complex feature extraction methods and fusion rules. There are no standard reference images and the publicly available infrared and visible image pairs are scarce. Most supervised DL-based solutions have to take pre-training on other labeled large datasets which may not behave well when testing. The few unsupervised fusion methods can hardly obtain ideal images with good visual impression. In this paper, an infrared and visible image fusion method based on unsupervised convolutional neural network is proposed. When designing the network structure, densely connected convolutional network (DenseNet) is used as the sub-network for feature extraction and reconstruction to ensure that more information of source images can be retained in the fusion images. As to loss function, the perceptual loss is creatively introduced and combined with the structure similarity loss to constrain the updating of weight parameters during the back propagation. The perceptual loss designed helps to improve the visual information fidelity (VIF) of the fusion image effectively. Experimental results show that this method can obtain fusion images with prominent targets and obvious details. Compared with other 7 traditional and deep learning methods, the fusion results of this method are better on objective evaluation and visual observation when taken together.