Weed interference is a major constraint in the production of Angelica dahurica, where seedling–weed similarity, small target size, soil background variation, and occlusion make reliable field recognition difficult. This study develops a lightweight attention-enhanced one-stage detector for real-time crop-versus-weed localization in A. dahurica fields. The detector combines a MobileNetV3 backbone for computationally efficient feature extraction, the parameter-free SimAM mechanism for discriminative feature enhancement, and the Inner-EIoU loss for improved localization of small and partially occluded targets. A field image dataset was collected in Luoning County, Luoyang City, Henan Province, China, comprising a total of 2286 original images. To avoid potential data leakage, the dataset was first divided into training, validation, and test sets at the original-image level. Data augmentation was then performed separately within each subset, resulting in a final dataset of 11,430 images. Under the augmented-test evaluation protocol, the proposed detector achieved 92.5% precision, 94.8% recall, 93.6% F1-score, and 95.2% mAP@0.5, while requiring 0.275 million parameters and 1.1 GFLOPs and reaching 215 FPS in offline inference. For an independent field-validation dataset collected after model training and deployment, precision, recall, F1-score, and mAP@0.5 were 91.6%, 88.4%, 90.0%, and 91.8%, respectively. On an NVIDIA Jetson Orin Nano 8GB edge platform, TensorRT FP16 acceleration produced an average speed of 46.8 FPS. The results show that the proposed detector offers a practical balance among detection accuracy, model compactness, inference speed, and field adaptability. Accordingly, it can serve as a visual-perception component for precision-weeding systems in Angelica dahurica fields.