In the past few years, structural health monitoring (SHM) has become an important technology to ensure the safety of structures. Structural damage identification methods based on machine learning techniques have gained wide attention due to the advantages of effectively extracting features from monitoring data. However, the existing machine learning-based methods heavily depend on manually selected feature parameters from raw signals. This will cause the selected feature to obtain the optimal solution for a specific condition but may fail to provide a similar performance in other cases. In addition, the feature selection takes a long time, which can fail to achieve real-time performance in a practical structure. To address these problems, this article proposes a hybrid deep learning framework for structural damage identification that includes three components, namely, ensemble empirical mode decomposition (EEMD), Pearson correlation coefficient (PCC), and a convolutional neural network (CNN). The proposed EEMD-PCC-CNN method is capable of automatically extracting features from raw signals to satisfy any damage identification objective. To evaluate the performance of the proposed EEMD-PCC-CNN method, a three-story building structure is investigated. The acceleration signal of the three-story building structure is first analyzed by EEMD. After obtaining the time-frequency information, PCC is utilized to select optimal time-frequency information as the input of the CNN for damage identification. Compared with other classical methods (SVM, KNN, RF, etc.), the experimental results show that the newly proposed EEMD-PCC-CNN method has significant performance advantages in damage identification. In addition, the accuracy of the proposed damage identification method is improved by more than 4% after utilizing EEMD in comparison with CNN alone.
In the process of bridge management, large amounts of domain information are accumulated, such as basic attributes, structural defects, technical conditions, etc. However, the valuable information is not fully utilized, resulting in insufficient knowledge service in the field of bridge management. To tackle these problems, this paper proposes a complex knowledge base question answering (C-KBQA) framework for intelligent bridge management based on multi-task learning (MTL) and cross-task constraints (CTC). First, with C-KBQA as the main task, part-of-speech (POS) tagging, topic entity extraction (TEE), and question classification (QC) as auxiliary tasks, an MTL framework is built by sharing encoders and parameters, thereby effectively avoiding the error propagation problem of the pipeline model. Second, cross-task semantic constraints are provided for different subtasks via POS embeddings, entity embeddings, and question-type embeddings. Finally, using template matching, relevant query statements are generated and interaction with the knowledge base is established. The experimental results show that the proposed model outperforms compared mainstream models in terms of TEE and QC on bridge management datasets, and its performance in C-KBQA is outstanding.
13 June 2024 in IEEE Transactions on Instrumentation and Measurement
In recent years, the evolution of convolutional neural networks (CNNs) has paved the way for the development of deep learning-based visual inspection techniques to identify damages in concrete bridges. However, the coexistence of diverse damage attributes within a single image complicates the comprehensive interpretation of these images during practical bridge inspections, posing a persistent challenge for existing single classification tasks. To address this issue, we introduce an innovative algorithm that synergizes Graph Convolutional Neural Network (GCN) and MaxViT (Multi-Axis Vision Transformer). Our approach leverages MaxViT to extract image features and subsequently models the co-occurrence correlation of labels. Additionally, we employ a graph convolutional neural network to extract features that capture the correlation between labels. Ultimately, we integrate these image features to enable the multi-label classification of damage images. To evaluate the efficacy of our method, we curated a multi-label damage classification dataset for concrete bridges, consisting of 2098 samples encompassing the four prevalent types of spall, rebar, corrosion, and crack observed during bridge inspections. Experimental results showcase that our proposed method achieves a mean Average Precision (mAP) of 99.37% when applied to real bridge inspection images. In terms of general evaluation metrics, our method attains an accuracy of 98.29%, precision of 96.28%, recall of 97.08%, and F1-score of 96.68%. These findings underscore the superior performance of our method compared to current advanced multi-label classification networks, demonstrating higher accuracy. An interpretable analysis of the classification feature maps generated by our model, utilizing the Grad-CAM algorithm, reveals that the model effectively directs attention to specific regions where damage is located. Consequently, our method holds promising engineering applications and has the potential to significantly enhance the accuracy of support provided for bridge inspection and maintenance endeavors.
In the field of automated concrete damage detection, deep learning methods face challenges in real-time performance and accuracy. This paper proposes a lightweight method for concrete bridge damage detection, combining an enhanced YOLOv5 model with channel pruning algorithms. Firstly, ShuffleNetv2 is used as the feature extraction backbone, reducing network parameters. Secondly, bi-directional feature pyramid network fusion improves accuracy for small object damage by merging feature maps with contextual information. The model is then trained with channel sparse regularization and pruned based on sparsity rates. Finally, fine-tuning identifies optimal pruning parameters. Evaluated on a dataset of 2468 high-definition images containing four types of concrete bridge damage, the model achieves 72.92% precision, 77.85% recall, and 78.75% mean average precision (mAP), with the pruned model achieving 76.14% mAP. The model size is reduced from 10.26 M to 4.24 M, meeting real-time, high-accuracy requirements with strong generalizability for bridge detection.
Automated damage segmentation for concrete bridges is a fundamental task in infrastructure maintenance, yet existing systems often depend heavily on large annotated datasets, which are costly and time-consuming to produce. This paper presents an innovative framework for concrete bridge damage segmentation, leveraging the Segment Anything Model (SAM) to reduce the reliance on extensive annotated data while enhancing segmentation accuracy and efficiency. Firstly, a SAM-guided mask generation network is introduced, which utilizes the SAM’s segmentation capabilities to generate supplementary supervision labels for damage segmentation. Then, a novel point-prompting strategy, incorporating saliency information, is proposed to refine SAM’s prompts, ensuring accurate mask generation for complex damage patterns. Next, a trainable semantic segmentation network is designed, integrating MambaVision and ResNet as dual backbones to capture multi-level features from concrete bridge damages. To fuse these features effectively, a Hierarchical Attention Fusion (HAF) mechanism is introduced. Finally, a Polarized Self-Attention (PSA) decoder is employed to improve segmentation precision. Experiments on a dataset of 10,000 concrete bridge images with box-level annotations achieved state-of-the-art performance, with an MIoU of 60.13%, PA of 74.02%, and MDice of 75.40%, outperforming existing segmentation models. In summary, this study improves the accuracy of concrete bridge damage segmentation through a series of innovative methods and strategies, and the concrete bridge damage segmentation algorithm opens up new horizons and directions.
1 January 2023 in International Journal of Robotics and Automation