Recent deep learning based image editing methods have achieved promising results for removing object in an image but fail to generate plausible results for removing large objects of complex nature, especially in facial images. The objective of this work is to remove mask objects in facial images. This problem is challenging because (1) most of the time facial masks cover quite a large region of face that even extends beyond the actual face boundary below chin, and (2) facial image pairs with and without mask object do not exist for training. We break the problem into two stages: mask object detection and image completion of the removed mask region. The first stage of our model automatically produces binary segmentation for the mask region. Then, the second stage removes the mask and synthesizes the affected region with fine details while retaining the global coherency of face structure. For this, we have employed a GAN-based network using two discriminators where one discriminator helps learn the global structure of the face and then another discriminator comes in to focus learning on the deep missing region. To train our model in a supervised manner, we create a paired synthetic dataset using publicly available CelebA dataset and evaluated on real world images collected from the Internet. Our model outperforms others representative state-of-the-art approaches both qualitatively and quantitatively.

Removing a specific object from an image and replacing the hole left behind with visually plausible backgrounds is a very intriguing task. While recent deep learning based object removal methods have shown promising results on this task for some structured scenes, none of them have addressed the problem of object removal in facial images. The objective of this work is to remove microphone object in facial images and fill hole with correct facial semantics and fine details. To make our solution practically useful, we present an interactive method called MRGAN, where the user roughly provides the microphone region. For filling the hole, we employ a Generative Adversarial Network based image-to-image translation approach. We break the problem into two stages: inpainter and refiner. The inpainter estimates coarse prediction by roughly filling in the microphone region followed by the refiner which produces fine details under the microphone region. We unite perceptual loss, reconstruction loss and adversarial loss as joint loss function for generating a realistic face and similar structure to the ground truth. Because facial image pairs with or without microphone do not exist, we have trained our method on a synthetically generated microphone dataset from CelebA face images and evaluated on real world microphone images. Our extensive evaluation shows that MRGAN performs better than state-of-the-art image manipulation methods on real microphone images although we only train our method using the synthetic dataset created. Additionally, we provide ablation studies for the integrated loss function and for different network arrangements.

This research features a user-friendly method for face de-occlusion in facial images where the user has control of which object to remove. Our system removes one object at a time, however, it is capable of removing multiple objects through repeated application. Although we show the effectiveness of our system on five commonly occurring occluding objects including hands, a medical mask, microphone, sunglasses, and eyeglasses, more types of object can be considered based on the proposed methodology. Our model learns to detect a user-selected, possibly distracting, object in the first stage. Then, the second stage removes the object using the object detection information from the first stage as guidance. To achieve this, we employ GAN-based networks in both stages. Specifically, in the second stage, we integrate both partial and vanilla convolution operations in the generator part of the GAN network. We show that by using this integration, the proposed network can learn a well-incorporated structure and also overcome the problem of visual discrepancies in the affected region of the face. To train our network, we produce a paired synthetic face-occluded dataset. Our model is evaluated using real world images collected from the Internet and publicly available CelebA and CelebA-HQ datasets. Experimental results confirm our model’s effectiveness in removing challenging foreground non-face objects from facial images as compared to the existing representative state-of-the-art approaches.

Methods that enable the visual inspection of solar panels are currently in demand, as a huge number of solar panels are now being deployed as a sustainable energy source. One of the solutions for inspection automation is an end-toend deep learning framework, but this is not recommended for this problem because such a framework requires not only powerful computational resources, but also a large-scale classbalanced dataset. In this study, we present a cost-effective solar panel defect detection method. We emphasize the spatial feature of defects by utilizing an attention map that is generated by a pre-trained attention mechanism that can give attention on stroke ends, gathering, and bends. We define and extract 13 statistical features from the attention map, and then feed them into conventional machine learning model. Therefore, we no longer require energy depleting models such as end-to-end neural classifiers to discriminate between non-defective and defective panels. Five conventional machine learning models and one stateof-the-art (SOTA) deep learning model—i. e., EfficientNet—are used to generalize the experimental results. The results of the comparative experiments indicate that our approach, which includes attention mechanism recycling and statistical feature extraction, is guaranteed to provide cost-effective defect detection in general with performance that is competitive with that of recent SOTA. In future research, we expect that our approach can be adopted in other defect detection tasks such as steel or film manufacturing processes.

Image mosaicing is an image processing technique that is most commonly used to conceal identities of sensitive objects. The authors’ research features recovering the mosaiced parts in an image, especially focusing on facial parts. While recent image completion methods based on deep learning have shown promising results on recovering damaged parts in an image, they have not addressed the problem of image unmosaicing. Moreover, all those methods necessitate the location information of damaged parts to tackle the recovery problem. They formulate unmosaicing as an image‐to‐image translation problem, and propose a two‐stage method using generative adversarial network (GAN): stage‐I GAN generates a coarse prediction followed by stage‐II GAN which produces a final unmosaiced image with finer information. A combination of low‐level loss and high‐level structural similarity loss is used to attain visually plausible and semantically consistent output. They have evaluated their method on the CelebA dataset and achieved better results than state‐of‐the‐art image completion methods without explicitly exploiting the location information of mosaiced parts.

Due to scarcity of anomaly situations in the early manufacturing stage, an unsupervised anomaly detection (UAD) approach is widely adopted which only uses normal samples for training. This approach is based on the assumption that the trained UAD model will accurately reconstruct normal patterns but struggles with unseen anomalies. To enhance the UAD performance, reconstruction-by-inpainting based methods have recently been investigated, especially on the masking strategy of suspected defective regions. However, there are still issues to overcome: (1) time-consuming inference due to multiple masking, (2) output inconsistency by random masking, and (3) inaccurate reconstruction of normal patterns for large masked areas. Motivated by this, this study proposes a novel reconstruction-by-inpainting method, dubbed Excision And Recovery (EAR), that features single deterministic masking based on the ImageNet pre-trained DINO-ViT and visual obfuscation for hint-providing. Experimental results on the MVTec AD dataset show that deterministic masking by pre-trained attention effectively cuts out suspected defective regions and resolves the aforementioned issues 1 and 2. Also, hint-providing by mosaicing proves to enhance the performance than emptying those regions by binary masking, thereby overcomes issue 3. The proposed approach achieves a high performance without any change of the model structure. Promising results are shown through laboratory tests with public industrial datasets. To suggest EAR be possibly adopted in various industries as a practically deployable solution, future steps include evaluating its applicability in relevant manufacturing environments.