As a key component of inland waterway infrastructure, the structural integrity of waterway revetments is directly related to navigation safety and aquatic ecological stability. However, in complex inland water environments, the process of damage detection and hazard prevention is confronted with numerous challenges. These include intense background interference, a high missed detection rate for small-scale damage, and insufficient real-time performance. To address the aforementioned issues, this paper proposes DH-YOLO, a lightweight architecture for damage detection. Building upon the efficient YOLOv8n backbone, the proposed model enhances feature sampling and discriminability capabilities through the task-specific integration of the Dynamic Upsampler (Dy_Sample) and the Hybrid Attention Transformer Head (HATHead) modules. This study emphasizes the detection of damage presence, rather than its classification or recognition. Evaluated on a custom-built image dataset of revetment damage, the proposed method achieves a Precision of 81.5%, Recall of 76.9%, mAP50 of 81.7%, and mAP50–95 of 48.6%, while maintaining a low computational cost of 6.9 GFLOPs. These results indicate that compared with mainstream models, the proposed algorithm achieves a favorable overall trade-off between accuracy and efficiency, and exhibits application potential for real-time damage detection and hazard prevention in complex inland waterway environments.