Medical image fusion (MIF) has received painstaking attention due to its diverse medical applications in response to accurately diagnosing clinical images. Numerous MIF methods have been proposed to date, but the fused image suffers from poor contrast, non-uniform illumination, noise presence, and improper fusion strategies, resulting in an inadequate sparse representation of significant features. This paper proposes the morphological preprocessing method to address the non-uniform illumination and noise by the bottom-hat–top-hat strategy. Then, grey-principal component analysis (grey-PCA) is used to transform RGB images into gray images that can preserve detailed features. After that, the local shift-invariant shearlet transform (LSIST) method decomposes the images into the low-pass (LP) and high-pass (HP) sub-bands, efficiently restoring all significant characteristics in various scales and directions. The HP sub-bands are fed to two branches of the Siamese convolutional neural network (CNN) by process of feature detection, initial segmentation, and consistency verification to effectively capture smooth edges, and textures. While the LP sub-bands are fused by employing local energy fusion using the averaging and selection mode to restore the energy information. The proposed method is validated by subjective and objective quality assessments. The subjective evaluation is conducted by a user case study in which twelve field specialists verified the superiority of the proposed method based on precise details, image contrast, noise in the fused image, and no loss of information. The supremacy of the proposed method is further justified by obtaining 0.6836 to 0.8794, 0.5234 to 0.6710, and 3.8501 to 8.7937 gain for , CRR, and AG and noise reduction from 0.3397 to 0.1209 over other methods for objective parameters.
Deep learning (DL) has achieved significant attention in the field of infrared (IR) and visible (VI) image fusion, and several attempts have been made to enhance the quality of the final fused image. It produces better results than conventional methods; however, the captured image cannot acquire useful information due to environments with poor lighting, fog, dense smoke, haze, and the noise generated by sensors. This paper proposes an adaptive fuzzy-based preprocessing enhancement method that automatically enhances the contrast of images with adaptive parameter calculation. The enhanced images are then decomposed into base and detail layers by anisotropic diffusion-based edge-preserving filters that remove noise while smoothing the edges. The detailed parts are fed into four convolutional layers of the VGG-19 network through transfer learning to extract features maps. These features maps are fused by multiple fusion strategies to obtain the final fused detailed layer. The base parts are fused by the PCA method to preserve the energy information. Experimental results reveal that our proposed method achieves state-of-the-art performance compared with existing fusion methods in a subjective evaluation through the visual experience of experts and statistical tests. Moreover, the objective assessment parameters are conducted by various parameters (, , , , , and ) which were used in the comparison method. The proposed method achieves 0.2651 to 0.3951, 0.5827 to 0.8469, 56.3710 to 71.9081, 4.0117 to 7.9907, and 0.6538 to 0.8727 gain for , , , , and , respectively. At the same time, the proposed method has more noise reduction (0.3049 to 0.0021) that further justifies the efficacy of the proposed method than conventional methods.
11 March 2024 in IEEE Transactions on Instrumentation and Measurement
With the development of smart grids, vision-based meter reading recognition methods are gradually replacing traditional manual inspections. In substations, meter reading faces image corruption challenges caused by various factors such as strong electromagnetic interference, lighting variations and weather conditions. The existing methods show significant performance degradation in noisy environments. To address this, we propose a robust pointer meter reading recognition method under image corruption. We first design image corruption augmentation (ICA), which significantly enhances the resilience of the model to disturbances. We introduce mask scoring convolution region-based convolutional neural network (MSC R-CNN) to segment pointer and scale masks on dial. MSC head improves localization and segmentation accuracy, while the balanced aggregation feature pyramid (BAFP) fuses features and enables multi-scale predictions. The inclusion of the global context (GC) block mitigates the impact of interference. For meter readings, we develop scale area proportion (SAP) reading method to process pointer and scale masks. The experimental results demonstrate that MSC R-CNN achieves 64.2 Mask mAP and 58.4 Mask mPC, surpassing the 58.4 Mask mAP and 26.0 Mask mPC of Mask R-CNN. SAP reading method achieves a successful meter reading rate (SMR rate) of 94.6% and maintains an average SMR rate of 92.4% under image corruption. The proposed method manifests robust meter reading recognition under image corruption. It can contribute to the further upgrading of smart grids. The code is available at https://github.com/ZhaolinWang0110/MeterRecognition.
Visual inspection technology based on deep learning has achieved great success in surface defect inspection tasks. Most existing works transfer the learned knowledge from the natural dataset ( e.g ., ImageNet dataset) into the target tasks. However, the paradigm is suboptimal for defect inspection tasks due to (1) the inherent dataset gap between natural and defect images and (2) the misalignment of task objectives. The limitations make the model cannot learn a generalized visual representation. To address the above issues, a contrastive and restorative self-supervised learning framework (CoRe) is proposed to learn general representation for the defect inspection task. Specifically, to bridge the dataset gap, we pre-train the model on the related dataset with a similar feature distribution instead of the natural dataset, which facilitates the representation learning of defect inspection tasks. Subsequently, to mitigate the task misalignment, the proposed method combines contrastive and restorative learning to excavate features required for the defect inspection task. The experimental results on five surface defect datasets demonstrate that our method outperforms the existing self-supervised learning works, typical supervised pre-training paradigm, and some specific defect inspection methods.
29 October 2021 in Neural Computing and Applications