In recent years, deep learning (DL) has been widely studied using various methods across the globe, especially with respect to training methods and network structures, proving highly effective in a wide range of tasks and applications, including image, speech, and text recognition. One important aspect of this advancement is involved in the effort of designing and upgrading neural architectures, which has been consistently attempted thus far. However, designing such architectures requires the combined knowledge and know-how of experts from each relevant discipline and a series of trial-and-error steps. In this light, automated neural architecture search (NAS) methods are increasingly at the center of attention; this paper aimed at summarizing the basic concepts of NAS while providing an overview of recent studies on the applications of NAS. It is worth noting that most previous survey studies on NAS have been focused on perspectives of hardware or search strategies. To the best knowledge of the present authors, this study is the first to look at NAS from a computer vision perspective. In the present study, computer vision areas were categorized by task, and recent trends found in each study on NAS were analyzed in detail.

Recently, thermal cameras are being widely used in various fields, such as intelligent surveillance, biometrics, and health monitoring. However, the high cost of the thermal cameras poses a challenge in terms of purchase. Additionally, thermal images have an issue pertaining to blurring caused by object movement, camera movement, and camera focus settings. There have been very few studies on image restoration centered around thermal images to address such problems. Moreover, it is important to increase the processing speed of image restoration methods to jointly conduct with methods such as action recognition and object tracking that use temporal information from thermal videos. However, no study has been conducted on simultaneously performing super-resolution reconstruction and deblurring using thermal images. Furthermore, existing studies on object detection using thermal images have errors owing to the incapability in distinguishing reflections on the surrounding ground or wall due to the heat radiated from the object. To address such issues, this study proposes a deep learning-based thermal image restoration method that simultaneously performs super-resolution reconstruction and deblurring. According to recent development of deep learning, generative adversarial network (GAN)-based methods which have ability to preserve texture details in images, and yield sharper and more plausible textures than classical feed forward encoders show success in image-to-image translation tasks. Considering the advantages of GAN, we propose a deblur-SRRGAN for thermal image reconstruction. In addition, we propose a light-weighted Mask R-CNN for object detection in the reconstructed thermal image. For the input, we employ an image processing method that converts 1-channel thermal images (often used in the existing studies) into 3-channel images. The results of the experiments conducted using self-collected databases and an open database demonstrate that our method outperforms the state-of-the-art methods.

Background: The early diagnosis of various gastrointestinal diseases can lead to effective treatment and reduce the risk of many life-threatening conditions. Unfortunately, various small gastrointestinal lesions are undetectable during early-stage examination by medical experts. In previous studies, various deep learning–based computer-aided diagnosis tools have been used to make a significant contribution to the effective diagnosis and treatment of gastrointestinal diseases. However, most of these methods were designed to detect a limited number of gastrointestinal diseases, such as polyps, tumors, or cancers, in a specific part of the human gastrointestinal tract.Objective: This study aimed to develop a comprehensive computer-aided diagnosis tool to assist medical experts in diagnosing various types of gastrointestinal diseases.Methods: Our proposed framework comprises a deep learning–based classification network followed by a retrieval method. In the first step, the classification network predicts the disease type for the current medical condition. Then, the retrieval part of the framework shows the relevant cases (endoscopic images) from the previous database. These past cases help the medical expert validate the current computer prediction subjectively, which ultimately results in better diagnosis and treatment.Results: All the experiments were performed using 2 endoscopic data sets with a total of 52,471 frames and 37 different classes. The optimal performances obtained by our proposed method in accuracy, F1 score, mean average precision, and mean average recall were 96.19%, 96.99%, 98.18%, and 95.86%, respectively. The overall performance of our proposed diagnostic framework substantially outperformed state-of-the-art methods.Conclusions: This study provides a comprehensive computer-aided diagnosis framework for identifying various types of gastrointestinal diseases. The results show the superiority of our proposed method over various other recent methods and illustrate its potential for clinical diagnosis and treatment. Our proposed network can be applicable to other classification domains in medical imaging, such as computed tomography scans, magnetic resonance imaging, and ultrasound sequences.

Iris recognition is one of the non-contact biometric identification methods that are hygienic and highly accurate. Iris recognition involves using iris images obtained by a near-infrared (NIR) camera or a visible light camera. A clear image of iris can be obtained when an NIR camera is used, but it requires an NIR illuminator in addition to the NIR camera. Iris recognition can be performed with a built-in camera device when a visible light camera is used, which also has the advantage of obtaining a three-channel image containing the color information. Accordingly, studies are being conducted on iris recognition by obtaining iris images from the face images taken by a high-resolution visible light camera in smartphones. However, when iris images have unconstrained conditions or are obtained without the cooperation of the subjects, the quality of iris images are reduced by noises such as optical and motion blur, off-angle view, specular reflection (SR), and other artifacts, thus ultimately deteriorating the recognition performance. Therefore, in this study, a method has been proposed for enhancing the quality of iris images by blurring the iris region and deep-learning-based deblurring. In addition, we propose the method for improving the recognition performance by integrating the recognition score in periocular regions and support vector machine (SVM). The method proposed in this study, which was experimented with noisy iris challenge evaluation-part II training database and MICHE database, exhibited an improved performance compared to the state-of-the-art methods.

Although various studies based on thermal images have been conducted, few studies have focused on the simultaneous extraction of joints and skeleton information of an object from a thermal image, and performed human action recognition using this information. Unlike in the case of visible light images, performing joint detection and skeleton generation on thermal images often leads to the complete disappearance of spatial information such as joints. In this case, it is extremely difficult to extract joints information from the object. Moreover, the accuracy of action recognition is significantly reduced owing to this issue. Therefore, a new method to extract joints and skeleton information is proposed in this study to address these issues. In the proposed method, an original 1-channel thermal image was converted into a 3-channel thermal image and then the images were combined to improve the extraction performance. A generative adversarial network (GAN) was used in the proposed method for extracting joints and skeleton information. In addition, research to recognize various human actions was conducted using the joints and skeleton information extracted by this method. The proposed human action recognition is performed by combining a convolutional neural network (CNN) and long short-term memory (LSTM). As a result of the experiments using self-collected and open data, it was found that the method proposed in this study shows good performance compared to other state-of-the-art methods.

Multimodal camera-based person re-identification (ReID) is important in the field of intelligent surveillance. Thermal cameras can solve the problem in that visible-light cameras cannot acquire the valid feature information of a person under poor illumination conditions. However, thermal cameras usually have lower frame resolution than visible-light cameras. To overcome this problem, we propose an adaptive selection of reconstructed input by generator or interpolation (AS-RIG) method, which can adaptively select the generative adversarial network (GAN), or an interpolation method (bi-linear or bi-cubic). AS-RIG automatically selects a resolution-model using the mean-squared error (MSE), feature distance (FD), and structural similarity (SSIM). To verify the performance of our proposed method, two open databases are used: the DBPerson-Recog-DB1 and Sun Yat-set University multiple modality Re-ID (SYSU-MM01). Infrared frames from both databases are resized to be smaller than the original ones for experimentation. Experimental results show that our generator outperforms traditional interpolation methods. In addition, the person ReID experimental results demonstrate that AS-RIG outperforms non-adaptive selection methods and state-of-the-art methods.