Accurate and rapid identification of toxic wild mushrooms is critical for food safety and public health, yet existing visual detection methods face severe limitations due to high interspecies morphological similarity, complex field backgrounds, and drastic scale variations. To address these challenges, we propose YOLO-SD, an improved multiscale detection model based on YOLOv11. A lightweight Simple Attention Module (SimAM) is embedded after the backbone’s final C2PSA layer to reweight deep semantic features and highlight discriminative regions. Meanwhile, the first C3k2 module in the neck is replaced by an optimized C3k2 with an improved Dilation-wise Residual Module (DWR) structure (denoted as C3k2_DWR), which adopts dual dilation branches, partial depthwise convolutions and a learnable scaling coefficient to enhance multiscale feature extraction with low parameter overhead. Ablation experiments verify the significant complementary effect between the two modules, whose combination achieves 0.883 mean Average Precision at IoU 0.5 (mAP@50), outperforming both single-module versions and the 0.865 YOLOv11 baseline. Comparative results show YOLO-SD reaches 89.3% Precision and 80.8% Recall, superior to mainstream YOLO variants while maintaining high inference efficiency. This work provides an effective intelligent identification solution for early warning of wild toxic mushroom hazards, contributing to agricultural ecological safety and food chain risk management.