20 February 2024 in IEEE Transactions on Instrumentation and Measurement
It is an urgent to monitor the dynamic industrial process in real-time, fast and accurately, and provide useful information for visualization. Various modalities include abundant information such as conflict, redundancy and complementarity. It is reasonable to combine different information from multiple modalities to make a more accurate assessment for image reconstruction. In this paper, a convolutional attention fusion network for electrical/ultrasound dual-modal fusion image reconstruction is proposed. Two fully connected modules are used to transform the measurement spatial information of different sensitive principles into image spatial information of the same dimension, which solves the heterogeneity problem of different modal information; in the process of feature extraction, the ill-posed of image reconstruction is improved by adding feature dimension; the spatial pyramid pooling module is used to obtain multi-scale feature information and provide more effective features for image reconstruction; according to the correlation between different modal features, the importance of different modal features to the reconstruction target is judged by attention mechanism, and different modal local features with different attention weights, which promotes the effective fusion and collaborative imaging of dual-modal information. The experimental results show that the convolution attention fusion reconstruction network improves the reconstruction accuracy and anti-noise performance compared with the dual-modal concatenation fusion reconstruction method and the single-modal reconstruction method.
21 August 2024 in IEEE Access
A convolutional concatenation fusion imaging network is proposed for Electrical resistance tomography (ERT) and ultrasonic transmission tomography (UTT) dual-modality fusion imaging, containing four stages: initial imaging, concatenation fusion, feature extraction, and image reconstruction. The fusion of heterogeneous sensors is completed through the initial imaging, and the measured voltage of ERT and the measured sound pressure of UTT are converted into pixel information with the same dimension. The feature information of ERT and UT is sequentially concatenated into the feature vector with the same scale to form a dual-modality shared semantic space, which realizes the fusion of dual-modality feature information. The feature extraction process consists of two convolution blocks and a spatial pyramid pooling block, which completes the self-mining and extraction of multiscale feature. The process of image reconstruction is composed of two deconvolution modules, which completes the reconstruction of multi-medium distribution on the basis of extracted feature. A set of numerical and experimental tests are carried out to evaluate the performance of the proposed method. The results show that the proposed ERT/UTT dual-modality imaging method has obvious improvement in imaging accuracy compared to the dual-modality methods WAF and single-modality methods ERT-CIN and UTT-CIN.