Neural network-based computer vision is widely used in industrial image detection due to the outstanding performance of fast and accurate defect recognition, which can be applied to the healthy recognition of Mini/Micro LED chips. However, limited by optical imaging of industrial cameras and chip physical size, the following challenges exist for Mini/Micro LED chip defect recognition: (1) The difference between a healthy chip and a defective chip image is small due to the limited size of the chip image, low image resolution and the few pixels occupied by the defect. (2) Standardized Mini/Micro LED industrial manufacturing produces a limited number of defective products, resulting in an imbalance of positive and negative samples. To overcome these challenges, a dual entropy-controlled convolutional neural network (DENC-CNN) combining feature entropy consistency (FEC) and gradient contribution entropy (GCE) is proposed. FEC is proposed to improve feature enrichment and consistency of information transfer to enhance the learnable of samples, with few fusion parameters. Global Attention Multi-scale Low-resolution Feature Fusion (GAMLF) is constructed and combined with FEC to retain the detail of multi-scale low-resolution features, enhancing the feature description capability of the model. To deal with the inevitable positive and negative sample imbalance, GCE is designed as part of the gradient weighting to guide the model to pay more attention on hard-to-classify samples, while avoiding over-focusing on hard-to-classify samples and over-fitting. We also construct a Mini/Micro LED dataset based on self-built image acquisition system to validate the proposed model. Experiments show that the proposed DENC-CNN achieves an accuracy of 99.12%, a G-mean of 97.86% and a F1-score of 97.87% for Mini/Micro LED chip defect recognition.
Vision-based unmanned aerial vehicle (UAV) localization is capable of providing real-time coordinates independently during GNSS interruption, which is important in security, agriculture, industrial mapping, and other fields. owever, there are problems with shadows, the tiny size of targets, interfering objects, and motion blurred edges in aerial images captured by UAVs. Therefore, a multi-order Siamese region proposal network (M-O SiamRPN) with weight adaptive joint multiple intersection over union (MIoU) loss function is proposed to overcome the above limitations. The normalized covariance of 2-O information based on1-O features is introduced in the Siamese convolutional neural network to improve the representation and sensitivity of the network to edges. We innovatively propose a spatial continuity criterion to select 1-O features with richer local details for the calculation of 2-O information, to ensure the effectiveness of M-O features. To reduce the effect of unavoidable positive and negative sample imbalance in target detection, weight adaptive coefficients were designed to automatically modify the penalty factor of cross-entropy loss. Moreover, the MIoU was constructed to constrain the anchor box regression from multiple perspectives. In addition, we proposed an improved Wallis shadow automatic compensation method to pre-process aerial images, providing the basis for subsequent image matching procedures. We also built a consumer-grade UAV acquisition platform to construct an aerial image dataset for experimental validation. The results show that our framework achieved excellent performance for each quantitative and qualitative metric, with the highest precision being 0.979 and a success rate of 0.732.
16 October 2024 in IEEE Transactions on Industrial Informatics
Mini/Micro LEDs are becoming the next generation of displays, with sizes shrinking to less than 100 μ m/50 μ m and integration scaling increasing by more than 100-fold. Fast and precise sorting, which includes the identification and localization of mass mini/micro LED chips, has become an urgent need in the industry due to its high efficiency and reliability. However, the small size, high density, and weak defects of the mini/micro LED, combined with the industrial rapid sorting requirement, pose challenges to the sorting task. To address these issues, we propose a fast and precise visual sorting approach that includes accurate chip identification and pixel-level localization. In detail, a double-ended self-attention (SA) encoder–decoder hybrid convolutional encoder neural network framework is designed to accommodate both local features and global information. Then, two generic patch enhancement modules are constructed to compensate for the local feature modeling inefficiencies inherent in SA. Finally, the light SA encoder (Li-SA Co) and light SA covariance decoder (Li-SA-Cov Dec) basic blocks are proposed to speed up sorting. The effectiveness of the proposed approach is demonstrated by the actual mini/micro LED industrial images acquired, which have an mAP of 97.1%, a localization success of 98.6%, and a speed of 20.1FPS, all of which are higher than those of other state-of-the-art methods.
In haptic recognition, pressure information is usually represented as an image, and then used for feature extraction and classification. Deep learning that processes haptic information in end-to-end manner has attracted attention. This study proposes a multiorder attentional spatial interactive convolutional neural network (MoAS-CNN) for haptic recognition. The asymmetric dual-stream all convolutional neural network with integrated channel attention module is applied for automatic first-order feature extraction. Later on, the spatial interactive features based on the overall feature map are computed to improve the second-order description capability. Finally, the multiorder features are summed to improve the feature utilization efficiency. To validate the MoAS-CNN, we construct a haptic acquisition platform based on three-scale pressure arrays and collect haptic letter-shape (A–Z) datasets with complex contours. The recognition accuracies are 95.73% for 16 × 16, 98.37% for 20 × 20 and 98.65% for 32 × 32, which significantly exceeds the traditional first- and second-order CNNs and local SIFT feature.