17 April 2019 in IEEE Transactions on Computational Imaging
Deep learning has found successful applications in restoration of 2D images including denoising, dehazing, superresolution and so on. However, existing DCNN architecture cannot fully exploit spatial-spectral correlations in 3D hyperspectral images (directly extending 2D DCNN into 3D will significantly increase computational complexity); meantime, unlike 2D images, there is an obstacle caused by the shortage of training data for hyperspectral images. To meet those challenges, we present a novel deep learning framework for 3D hyperspectral image denoising with the following contributions. First, inspired by the success of U-net in low-dose CT denoising, we propose a novel approach of encoding rich multi-scale information of hyperspectral images by a modified 3D U-net. Second, we present a computationally efficient implementation of 3D U-net based on the strategy of separable filtering. By decomposing 3D filtering into 2D spatial filtering and 1D spectral filtering, we can achieve substantial savings on the number of network parameters to keep computational complexity low. Third, we have developed a transfer learning approach of synthetically generating hyperspectral images from RGB images as supplementary training data. The synthesized hyperspectral images are used for initial training of modified 3D U-net denoising network which will be fine-tuned on real HSI images. Experimental results have shown that the proposed 3D U-net denoising method significantly outperforms existing model-based HSI denoising methods.