Automated computer vision-based inspections of railway infrastructures, such as component type, damaged status, and location, have been investigated actively by resorting to task-specific deep learning models. However, task-specific models that fulfill these separate inspection tasks encountered bottlenecks in improving inference accuracy, and bring huge computational costs. Multi-task deep learning, which can fulfill these inspection tasks concurrently, has yet to be fully investigated in the context of structural inspection. In this study, a multi-scale task interaction deep learning strategy is presented towards component recognition, damage segmentation, and depth estimation for a comprehensive post-earthquake inspection of high-speed railway viaducts. Three modules for multi-scale task interaction were proposed to modify the multi-task deep neural network, taking full advantage of task commonalities at multiple scales. The proposed method was validated with a large-scale image dataset of high-speed railway viaducts. Component recognition and depth estimation were incorporated to implement multi-task learning since they have higher pattern affinities at multiple scales. Results reported that mean Intersection over Unions of testing samples of component and damage tasks were 91.2% and 72.1%, RMSE of depth estimation was 1.54 m. Compared with single-task cases, training time, inference duration, and FLOPs of the multi-task model were reduced by 23%, 30%, 27% respectively. Results showed improvement in both inference accuracy and training efficiency, substantiating the superiority of the proposed strategy.
Component identification and depth estimation are important for detecting the integrity of post-disaster structures. However, traditional manual methods might be time-consuming, labor-intensive, and influenced by subjective judgments of inspectors. Deep-learning-based image visual inspection is a new approach to overcome these problems, but repeated modeling is required for different inspection tasks, which limits inspection accuracy and practical deployment efficiency. In this study, it is observed that the matched ratios of pixel pairs between component identification and depth estimation reach a high value, which indicates the dual tasks are highly related. Therefore, the Multi-Scale Task Interaction Network (MTI-Net) is proposed for structural images to simultaneously accomplish both tasks for accurate and efficient structural inspection. It propagates distilled task information from lower to higher scales. Then, it aggregates the refined task features from all scales to produce the final per-task predictions. Experiments show that MTI-Net delivers the full potential of multi-task learning, with a smaller memory footprint and higher efficiency compared to single-task learning. For the evaluation metrics of model performance, the mean Intersection over Union (mIoU) of component identification improves by 2.30, and root mean square error (RMSE) drops by 0.36 m with the aid of the multi-task strategy. The multi-task deep learning framework has great potential value in engineering applications.