23 October 2023 in IEEE Access
In industrial areas where anomalous samples are lacking, unsupervised methods with high accuracy are especially important to ensure product quality and stability. An unsupervised method based on a feature extractor and a distribution estimation module has been applied in the industrial field and has achieved good performance. Flow models are usually used as distribution estimation modules. But traditional flow models only focus on feature information in two dimensions, the channel dimension, and the space dimension, while ignoring the cross-dimensional connection between them. To solve this problem, we proposed a flow model with a global attention mechanism for anomaly detection of images. It embeds combined modules of global attention mechanism and convolutional layer in the structure of a reversible neural network, which is capable of global cross-dimensional extraction of image features. In the comparison experiments, our method achieves average image and pixel-level AUC of 0.997 and 0.987 on the MVTec AD dataset and 0.968 and 0.984 on the BTAD dataset, respectively, which outperforms the other traditional methods. In addition, in the detection task, our method also possesses faster inference speeds. This shows that our method has excellent anomaly detection and localization performance, which meets the industrial demand for high-precision anomaly detection and localization.
8 October 2021 in Security and Communication Networks
Recently, pretrained language models, such as Bert and XLNet, have rapidly advanced the state of the art on many NLP tasks. They can model implicit semantic information between words in the text. However, it is solely at the token level without considering the background knowledge. Intuitively, background knowledge influences the efficacy of text understanding. Inspired by this, we focus on improving model pretraining by leveraging external knowledge. Different from recent research that optimizes pretraining models by knowledge masking strategies, we propose a simple but general method to transfer explicit knowledge with pretraining. To be specific, we first match knowledge facts from a knowledge base (KB) and then add a knowledge injunction layer to a transformer directly without changing its architecture. This study seeks to find the direct impact of explicit knowledge on model pretraining. We conduct experiments on 7 datasets using 5 knowledge bases in different downstream tasks. Our investigation reveals promising results in all the tasks. The experiment also verifies that domain-specific knowledge is superior to open-domain knowledge in domain-specific task, and different knowledge bases have different performances in different tasks.