Recently, thermal cameras are being widely used in various fields, such as intelligent surveillance, biometrics, and health monitoring. However, the high cost of the thermal cameras poses a challenge in terms of purchase. Additionally, thermal images have an issue pertaining to blurring caused by object movement, camera movement, and camera focus settings. There have been very few studies on image restoration centered around thermal images to address such problems. Moreover, it is important to increase the processing speed of image restoration methods to jointly conduct with methods such as action recognition and object tracking that use temporal information from thermal videos. However, no study has been conducted on simultaneously performing super-resolution reconstruction and deblurring using thermal images. Furthermore, existing studies on object detection using thermal images have errors owing to the incapability in distinguishing reflections on the surrounding ground or wall due to the heat radiated from the object. To address such issues, this study proposes a deep learning-based thermal image restoration method that simultaneously performs super-resolution reconstruction and deblurring. According to recent development of deep learning, generative adversarial network (GAN)-based methods which have ability to preserve texture details in images, and yield sharper and more plausible textures than classical feed forward encoders show success in image-to-image translation tasks. Considering the advantages of GAN, we propose a deblur-SRRGAN for thermal image reconstruction. In addition, we propose a light-weighted Mask R-CNN for object detection in the reconstructed thermal image. For the input, we employ an image processing method that converts 1-channel thermal images (often used in the existing studies) into 3-channel images. The results of the experiments conducted using self-collected databases and an open database demonstrate that our method outperforms the state-of-the-art methods.

Automatic recognition of fake banknotes is an important task in practical banknote handling. Research on this task has mostly involved methods applied to automatic sorting machines with multiple imaging sensors or that use specialized sensors for capturing banknote images in various light wavelengths. These approaches can make use of the security features on banknotes for counterfeit detection. However, they require specialized devices, which are not always available for general users or visually impaired people. Meanwhile, smartphones are becoming more popular and can be useful imaging devices. Moreover, the types of fake banknotes created by imaging devices such as smartphone cameras or scanners are sometimes cannot be recognized by especially the visually impaired people. Addressing these problems, we propose a method for classifying fake and genuine banknotes using visible-light images captured by smartphone cameras based on convolutional neural networks. Experimental results on a self-collected dataset of US dollar, Euro, Korean won, and Jordanian dinar banknotes showed that our method performs better in terms of fake detection than the state-of-the-art methods.

A high-resolution thermal camera is very expensive and is thus difficult to be used. Furthermore, thermal images become blurred in various cases of object motion, camera shaking, and camera defocusing. To solve these problems, a previous super-resolution restoration (SRR) technique converting a thermal image acquired by a low-resolution camera into a high-resolution one, and a thermal image deblurring method have been researched. However, existing studies were performed based on 1-channel (grayscale) images. In addition, a large-sized and whole image has been used in the existing thermal image deblurring methods, which causes lower deblurring performance. In this study, we propose novel SRR and deblurring methods. The proposed deblurring method is conducted based on small region images. The proposed methods are also conducted using 3-channel (color) thermal images and generative adversarial networks. In addition, the performances of this method are compared in various color spaces (RGB, Gray, HLS, HSV, Lab, Luv, XYZ, YCrCb), image sizes, and thermal databases. Through experiments using self-collected databases and open databases, it was confirmed that the proposed methods show better performance than the state-of-the-art methods.

Extensive research has been conducted on image augmentation, segmentation, detection, and classification based on plant images. Specifically, previous studies on plant image classification have used various plant datasets (fruits, vegetables, flowers, trees, etc., and their leaves). However, existing plant-based image datasets are generally small. Furthermore, there are limitations in the construction of large-scale datasets. Consequently, previous research on plant classification using small training datasets encountered difficulties in achieving high accuracy. However, research on plant image classification based on small training datasets is insufficient. Accordingly, this study performed classification by reducing the number of training images of plant-image datasets by 70%, 50%, 30%, and 10%, respectively. Then, the number of images was increased back through augmentation methods for training. This ultimately improved the plant-image classification performance. Based on the respective preliminary experimental results, this study proposed a plant-image classification convolutional neural network (PI-CNN) based on plant image augmentation using a plant-image generative adversarial network (PI-GAN). Our proposed method showed the higher classification accuracies compared to the state-of-the-art methods when the experiments were conducted using four open datasets of PlantVillage, PlantDoc, Fruits-360, and Plants.

Although various studies based on thermal images have been conducted, few studies have focused on the simultaneous extraction of joints and skeleton information of an object from a thermal image, and performed human action recognition using this information. Unlike in the case of visible light images, performing joint detection and skeleton generation on thermal images often leads to the complete disappearance of spatial information such as joints. In this case, it is extremely difficult to extract joints information from the object. Moreover, the accuracy of action recognition is significantly reduced owing to this issue. Therefore, a new method to extract joints and skeleton information is proposed in this study to address these issues. In the proposed method, an original 1-channel thermal image was converted into a 3-channel thermal image and then the images were combined to improve the extraction performance. A generative adversarial network (GAN) was used in the proposed method for extracting joints and skeleton information. In addition, research to recognize various human actions was conducted using the joints and skeleton information extracted by this method. The proposed human action recognition is performed by combining a convolutional neural network (CNN) and long short-term memory (LSTM). As a result of the experiments using self-collected and open data, it was found that the method proposed in this study shows good performance compared to other state-of-the-art methods.

In vivo diseases such as colorectal cancer and gastric cancer are increasingly occurring in humans. These are two of the most common types of cancer that cause death worldwide. Therefore, the early detection and treatment of these types of cancer are crucial for saving lives. With the advances in technology and image processing techniques, computer-aided diagnosis (CAD) systems have been developed and applied in several medical systems to assist doctors in diagnosing diseases using imaging technology. In this study, we propose a CAD method to preclassify the in vivo endoscopic images into negative (images without evidence of a disease) and positive (images that possibly include pathological sites such as a polyp or suspected regions including complex vascular information) cases. The goal of our study is to assist doctors to focus on the positive frames of endoscopic sequence rather than the negative frames. Consequently, we can help in enhancing the performance and mitigating the efforts of doctors in the diagnosis procedure. Although previous studies were conducted to solve this problem, they were mostly based on a single classification model, thus limiting the classification performance. Thus, we propose the use of multiple classification models based on ensemble learning techniques to enhance the performance of pathological site classification. Through experiments with an open database, we confirmed that the ensemble of multiple deep learning-based models with different network architectures is more efficient for enhancing the performance of pathological site classification using a CAD system as compared to the state-of-the-art methods.