This study focused on the problem of domain shift in deep learning-based bridge element segmentation. The impracticability of accounting for all possible variabilities vis-à-vis structural shape, size, color, texture, illumination, and other operational conditions in the training process leads to the deterioration in the model performance when applied to test data from novel unseen domains. In such situations, rebuilding the model with labeled training data from the target domain becomes prohibitively expensive and time-consuming in many practical cases. Recent advancements in unsupervised domain adaptation techniques are known to provide viable solutions to this problem. However, it was observed in this study that the performance gain afforded by the domain adaptation techniques is not significant enough to adequately close the domain gaps commonly encountered in vision-based robotic bridge inspections. This study, therefore, proposed a class-wise histogram matching-based data augmentation technique that seeks to complement the domain adaptation strategy, leading to a significantly improved adaptation in situations where no labeled data are available from the target domain. The proposed framework is validated with two case studies concerning deep learning-based bridge element segmentation in inspection images collected by unmanned aerial vehicles (UAVs). It produced a mean intersection-over-union which is 21.2% and 21.3% higher than a benchmark domain adaptation method. In the future, this study can be extended to other relevant application areas, including but not limited to autonomous vision-based bridge defect detection and post-disaster structural reconnaissance.