In machine vision-based surface inspection tasks, defects are typically considered as local anomalies in homogeneous background. However, industrial workpieces commonly contain complex structures, including hallow regions, welding joints, or rivet holes. Such obvious structural interference will inevitably cause cluttered background and mislead the classification results. Moreover, the sizes of various surface defects might change significantly. Last but not the least, it is extremely time-consuming and not scalable to capture large-scale defect datasets to train deep CNN models. To address the challenges mentioned above, we firstly proposed to incorporate multiple convolutional layers with different kernel sizes to increase the receptive field and to generate multi-scale features. As a result, the proposed model can better handle cluttered background and defects of various sizes. Also, we purposely compress the size of parameters in the newly added convolutional layers for better learning of defect-related features using a limited number of training samples. Evaluated in a newly constructed surface defect dataset (images contain complex structures and defects of various sizes), our proposed model achieves more accurate recognition results compared with the state-of-the-art surface defect classifiers. Moreover, it is a light-weight model and can deliver real-time processing speed (>100fps) on a computer equipped with a single NVIDIA TITAN X Graphics Processing Unit (12G memory).

Fusion of images acquired using different sensors generates a single output with enhanced information for high-level visual perception applications. The transformer architecture has demonstrated its powerful ability to obtain important global contextual dependencies for multi-modal image fusion tasks. However, transformer-based image fusion methods face many critical issues, such as incurring huge computational burdens, limited ability to learn local features, and the difficulty of handling images of arbitrary sizes. To address the above limits, we proposed a novel Laplacian Pyramid Hybrid (LapH) network to combine the advantages of CNN and transformer architectures for multi-modal image fusion tasks. With the divide-and-conquer philosophy, we first build a light-weight CNN-based branch, performing effective extraction and fusion of texture/edge features via central difference convolutions, to process the high-resolution components with abundant details encoded in the lower pyramid levels of the Laplacian pyramid. Then, we design a transformer-based branch to process the low-resolution base components, learning long-range dependencies of global-contextual features without incurring extensive computational loads. Here, we design a multi-scale recurrent modulation mechanism to integrate the edge/texture features from the CNN branch as guidance to progressively refine the feature extraction and fusion on low-frequency components. Finally, we propose a new multi-scale spatial consistency loss term based on the neighbor contrast in source images, generating fused images with more natural and realistic appearances. Extensive experiments on two different multi-modal image fusion tasks verify the superiority of our method. The source codes are made publicly available at https://github.com/rgttadv/LapH.

Surface defect detection is an important technique to realize product quality inspection. In this study, we develop an innovative multi-scale pooling convolutional neural network to accomplish high-accuracy steel surface defect classification. The model was built based on SqueezeNet, and experiments were carried out on the NEU noise-free and noisy testing set. Class activation map visualization proves that the multi-scale pooling model can accurately capture the defect location at multiple scales, and the defect feature information at different scales can complement and reinforce each other to obtain more robust results. Through T-SNE visualization analysis, it is found that the classification results of this model have large inter-class distance and small intra-class distance, indicating that this model has high reliability and strong generalization ability. In addition, the model is small in size (3MB) and runs at up to 130FPS on an NVIDIA 1080Ti GPU, making it suitable for applications with high real-time requirements.