This study introduces a non-invasive approach to monitor operation and productivity of a legacy pipe bending machine in real-time based on a lightweight convolutional neural network (CNN) model and internal sound as input data. Various sensors were deployed to determine the optimal sensor type and placement, and labels for training and testing the CNN model were generated through the meticulous collection of sound data in conjunction with webcam videos. The CNN model, which was optimized through hyperparameter tuning via grid search and utilized feature extraction using Log-Mel spectrogram, demonstrated notable prediction accuracies in the test. However, when applied in a real-world manufacturing scenario, the model encountered a significant number of errors in predicting productivity. To navigate through this challenge and enhance the predictive accuracy of the system, a buffer algorithm using the inferences of CNN models was proposed. This algorithm employs a queuing method for continuous sound monitoring securing robust predictions, refines the interpretation of the CNN model inferences, and enhances prediction outcomes in actual implementation where accuracy of monitoring productivity information is crucial. The proposed lightweight CNN model alongside the buffer algorithm was successfully deployed on an edge computer, enabling real-time remote monitoring.

Vision-based robots have been utilized for pick-and-place operations by their ability to find object poses. As they progress into handling a variety of objects with cluttered state, more flexible and lightweight operations have been presented. In this paper, an autonomous robotic bin-picking platform is proposed. It combines human demonstration with a collaborative robot for the flexibility of the objects and YOLOv5 neural network model for faster object localization without prior computer-aided design models or dataset in the training. After a simple human demonstration of which target object to pick and place, the raw color and depth images were refined, and the one on top of the bin was utilized to create synthetic images and annotations for the YOLOv5 model. To pick up the target object, the point cloud was lifted using the depth data corresponding to the result of the trained YOLOv5 model, and the object pose was estimated by matching them with Iterative Closest Points (ICP) algorithm. After picking up the target object, the robot placed it where the user defined it in the previous human demonstration stage. From the result of experiments with four types of objects and four human demonstrations, it took a total of 0.5 s to recognize the target object and estimate the object pose. The success rate of object detection was 95.6%, and the pick-and-place motion of all the found objects was successful.

An instantaneous and precise coating inspection method is imperative to mitigate the risk of flaws, defects, and discrepancies on coated surfaces. While many studies have demonstrated the effectiveness of automated visual inspection (AVI) approaches enhanced by computer vision and deep learning, critical challenges exist for practical applications in the manufacturing domain. Computer vision has proven to be inflexible, demanding sophisticated algorithms for diverse feature extraction. In deep learning, supervised approaches are constrained by the need for annotated datasets, whereas unsupervised methods often result in lower performance. Addressing these challenges, this paper proposes a novel deep learning-based automated visual inspection (AVI) framework designed to minimize the necessity for extensive feature engineering, programming, and manual data annotation in classifying fuel injection nozzles and discerning their coating interfaces from scratch. This proposed framework comprises six integral components: It begins by distinguishing between coated and uncoated nozzles through gray level co-occurrence matrix (GLCM)-based texture analysis and autoencoder (AE)-based classification. This is followed by cropping surface images from uncoated nozzles, and then building an AE model to estimate the coating interface locations on coated nozzles. The next step involves generating autonomously annotated datasets derived from these estimated coating interface locations. Subsequently, a convolutional neural network (CNN)-based detection model is trained to accurately localize the coating interface locations. The final component focuses on enhancing model performance and trustworthiness. This framework demonstrated over 95% accuracy in pinpointing the coating interfaces within the error range of ± 6 pixels and processed at a rate of 7.18 images per second. Additionally, explainable artificial intelligence (XAI) techniques such as t-distributed stochastic neighbor embedding (t-SNE) and the integrated gradient substantiated the reliability of the models.

In this research, we propose an Error Separation Technique (EST) based on optical fiber sensors for on-machine straightness error measurement. Two fiber optic Fabry–Pérot interferometers have been developed serving as two displacement sensors. The displacement distance is computed according to the reflected spectrum from interferometers, which can achieve a sub-micrometer resolution. The two-point method has been employed to separate the straightness error of the slides and the profile error of a fine-polished standard block. The spacing distance between two interferometers is determined by the diameter of optical fibers so that the EST's resolution has the potential to reach the sub-millimeter scale. In the experiment, the straightness error has been measured on a commercially available computer numerical control machine tool, and the measurement has been conducted on its x-axis. The spacing distance between two optical fiber sensors is 1.5 mm which equals the EST's resolution along the machine tool's x-axis. The separated profile error of the measured standard block is around 30 µm which has been verified by a high precision Coordinate Measurement Machine (CMM). The magnitude of the separated straightness error is around 40 µm. This technique is flexible and simple to be conducted, which can contribute to the micro-machine tool calibration and other straightness error applications.