Fusion of images acquired using different sensors generates a single output with enhanced information for high-level visual perception applications. The transformer architecture has demonstrated its powerful ability to obtain important global contextual dependencies for multi-modal image fusion tasks. However, transformer-based image fusion methods face many critical issues, such as incurring huge computational burdens, limited ability to learn local features, and the difficulty of handling images of arbitrary sizes. To address the above limits, we proposed a novel Laplacian Pyramid Hybrid (LapH) network to combine the advantages of CNN and transformer architectures for multi-modal image fusion tasks. With the divide-and-conquer philosophy, we first build a light-weight CNN-based branch, performing effective extraction and fusion of texture/edge features via central difference convolutions, to process the high-resolution components with abundant details encoded in the lower pyramid levels of the Laplacian pyramid. Then, we design a transformer-based branch to process the low-resolution base components, learning long-range dependencies of global-contextual features without incurring extensive computational loads. Here, we design a multi-scale recurrent modulation mechanism to integrate the edge/texture features from the CNN branch as guidance to progressively refine the feature extraction and fusion on low-frequency components. Finally, we propose a new multi-scale spatial consistency loss term based on the neighbor contrast in source images, generating fused images with more natural and realistic appearances. Extensive experiments on two different multi-modal image fusion tasks verify the superiority of our method. The source codes are made publicly available at https://github.com/rgttadv/LapH.

The inspection of welding surface quality is an important task for welding work. With the development of product quality inspection technology, automated and machine vision-based inspection have been applied to more industrial application fields because of its non-contact, convenience, and high efficiency. However, challenging material and optical phenomena such as high reflective surface areas often present on welding seams tend to produce artifacts such as holes in the reconstructed model using current visual sensors, hence leading to insufficiency or even errors in the inspection result. This paper presents a 3D reconstruction technique for highly reflective welding surfaces based on binocular style structured light stereo vision. The method starts from capturing a fully lit image for identifying highly reflective regions on a welding surface using conventional computer vision models, including gray-scale, binarization, dilation, and erosion. Then, fringe projection profilometry is used to generate point clouds on the interested area. The mapping and alignment from 2D image to 3D point cloud is then established to highlight features that are vital for eliminating “holes”—large featureless areas—caused by high reflections such as the specular mirroring effect. A two-way slicing method is proposed to operate on the refined point cloud, following the concept of dimensionality reduction to project the sliced point cloud onto different image planes before a Smoothing Spline model is applied to fit the discrete point formed by projection. The 3D coordinate values of points in the “hole” region are estimated according to the fitted curves and appended to the original point cloud using iterative algorithms. Experiment results verify that the proposed method can accurately reconstruct a wide range of welding surfaces with significantly improved precision.

Surface defect detection is crucial in industrial production, and due to the conveyor speed, real-time detection requires 30–60 frames per second (FPS), which exceeds the capability of most existing methods. This demand for high FPS has driven the need for lightweight detection models. Despite significant advancements in deep learning-based detection that have enabled single-stage models such as the you only look once (YOLO) series to achieve relatively fast detection, existing methods still face challenges in detecting multi-scale defects and tiny defects on complex surfaces while maintaining detection speed. This study proposes a lightweight single-stage detection model called lightweight industrial defect detection network with improved YOLO architecture (LIDD-YOLO) for high-precision and real-time industrial defect detection. Firstly, we propose the large separable kernel spatial pyramid pooling (SPP) module, which is a SPP structure with a separable large kernel attention mechanism, significantly improving the detection rate of multi-scale defects and enhancing the detection rate of small target defects. Secondly, we improved the Backbone and Neck structure of YOLOv8n with dual convolutional (Dual Conv) kernel convolution and enhanced the faster implementation of Cross Stage Partial Bottleneck with 2 Convolutions (C2f) module in the Neck structure with ghost convolution and decoupled fully connected (DFC) attention, reducing the computational and parameter overhead of the model while ensuring detection accuracy. Experimental results on the NEU-DET steel defect datasets and printed circuit board (PCB) defect datasets demonstrate that compared to YOLOv8n, LIDD-YOLO improves the recognition rate of multi-scale defects and small target defects while meeting lightweight requirements. LIDD-YOLO achieves a 3.2% increase in mean average precision (mAP) on the NEU-DET steel defect dataset, reaching 79.5%, and a 2.6% increase in mAP on the small target PCB defect dataset, reaching 93.3%. Moreover, it reduces the parameter count by 20.0% and floating point operations by 15.5%, further meeting the requirements for lightweight and high-precision industrial defect detection models.