12 December 2019 in IEEE Access
We present a novel privacy-preserving scheme for deep neural networks (DNNs) that enables us not to only apply images without visual information to DNNs but to also consider the use of independent encryption keys for both training and testing images for the first time. In this paper, a novel pixel-based image encryption method that maintains important features of original images is proposed for privacy-preserving DNNs. For training, a DNN model is trained with images encrypted by using the proposed method with independent encryption keys. For testing, the model enables us to apply both encrypted images and plain images for image classification. Therefore, there is no need to manage keys. In addition, the proposed method allows us to perform data augmentation in the encrypted domain. In an experiment, the proposed method is applied to well-known networks, that is, deep residual networks and densely connected convolutional networks, for image classification. The experimental results demonstrate that the proposed method, under the use of independent encryption keys, can maintain a high classification performance, and it is robust against ciphertext-only attacks (COAs). Moreover, the results confirm that the proposed scheme is able to classify plain images as well as encrypted images, even when data augmentation is carried out in the encrypted domain.
20 March 2019 in IEEE Transactions on Image Processing
We propose a novel method for adjusting luminance for multi-exposure image fusion. For the adjustment, two novel scene segmentation approaches based on luminance distribution are also proposed. Multi-exposure image fusion is a method for producing images that are expected to be more informative and perceptually appealing than any of the input ones, by directly fusing photos taken with different exposures. However, existing fusion methods often produce unclear fused images when input images do not have a sufficient number of different exposure levels. In this paper, we point out that adjusting the luminance of input images makes it possible to improve the quality of the final fused images. This insight is the basis of the proposed method. The proposed method enables us to produce high-quality images, even when undesirable inputs are given. Visual comparison results show that the proposed method can produce images that clearly represent a whole scene. In addition, multi-exposure image fusion with the proposed method outperforms state-of-the-art fusion methods in terms of MEF-SSIM, discrete entropy, tone mapped image quality index, and statistical naturalness.
22 April 2021 in IEEE Access
We propose a novel image transformation network for generating visually protected images for privacy-preserving deep neural networks (DNNs). The proposed transformation network is trained by using a plain image dataset so that plain images are converted into visually protected ones. Conventional perceptual encryption methods cause some accuracy degradation in image classification and are not robust enough against state-of-the-art attacks. In contrast, the proposed network not only enables us to maintain the image classification accuracy that using plain images achieves but is also strongly robust against attacks including DNN-based ones. Furthermore, there is no need to manage any security keys as the conventional methods require. In an image classification experiment, the proposed network is demonstrated to strongly protect the visual information of plain images while maintaining a high classification accuracy under the use of two typical classification networks: ResNet and VGG. In addition, it is shown that the visually protected images are robust enough against various attacks in an experiment in which we tried to restore the visual information of plain images.
8 January 2020 in IEEE Access
In this paper, we propose a novel hue-correction scheme based on the constant-hue plane in the RGB color space for deep-learning-based color-image enhancement. Existing hue-preserving image-enhancement methods cannot be applied to state-of-the-art enhancement methods such as deep-learning-based ones. Our main contributions are a discussion on the enhancement performance and the hue distortion of existing image-enhancement methods as well as the first attempt to make hue correction applicable to any existing image-enhancement methods including deep-learning-based ones. This novel scheme is carried out on the basis of the constant-hue plane in the RGB color space. In simulations, we first evaluated conventional image-enhancement methods in terms of the enhancement performance and hue distortion by using five objective metrics: the maximally saturated color similarity, the hue difference in CIEDE2000, discrete entropy, HIGRADE, and NIQMC. The experimental results show that recent deep-learning-based methods have a higher enhancement performance but cause images to be hue-distorted. In addition, the proposed scheme is demonstrated to be effective for suppressing hue distortion even under the use of deep-learning-based enhancement methods. Furthermore, it allows us not only to correct hue but also to maintain the performance of image-enhancement methods.
27 May 2019 in IEEE Access
In this paper, we propose a novel inverse tone mapping network, called “iTM-Net.” For training iTM-Net, we also propose a novel loss function that considers the non-linear relation between low dynamic range (LDR) and high dynamic range (HDR) images. For inverse tone mapping with convolutional neural networks (CNNs), we first point out that training CNNs with a standard loss function causes a problem due to the non-linear relation between LDR and HDR images. To overcome the problem, the novel loss function non-linearly tone-maps target HDR images into LDR ones on the basis of a tone mapping operator, and the distance between the tone-mapped images and predicted ones is then calculated. The proposed loss function enables us not only to normalize HDR images but also to reduce the non-linear relation between LDR and HDR ones. Experimental results show that HDR images predicted by the proposed iTM-Net have higher-quality than HDR ones predicted by conventional inverse tone mapping methods including the state-of-the-art, in terms of both HDR-VDP-2.2 and PU encoding + MS-SSIM. In addition, compared with loss functions that do not consider the non-linear relation, the proposed loss function is shown to improve the performance of CNNs.
19 August 2022 in IEEE Access
In this paper, we propose a novel white balance adjustment for multi-illuminant scenes, called “N-white balancing,” in which N source white points are mapped into a ground truth one. Most white balance adjustments focus on adjusting single-illuminant scenes. Several state-of-the-art methods for adjusting multi-illuminant scenes have been proposed, but they need to know the number of segments or the number of light sources in advance. Multi-color balance adjustments have been investigated to improve the performance of white balancing, but they also suffer from color distortion due to the rank deficiency problem. In contrast, the proposed method, N-white balancing, can correct multi-illuminant scenes even when we do not know the exact number of segments or light sources in a scene. In an experiment, the proposed method was demonstrated to outperform state-of-the-art methods under various illumination conditions such as single and multiple illuminants including non-uniform light sources.