Unmanned aerial vehicle (UAV) remote sensing has become an important tool for high-resolution tree species identification in orchards and forests. However, irregular spatial distribution, overlapping canopies, and small crown sizes still limit detection accuracy. To overcome these challenges, we propose YOLOv11-OAM, an enhanced one-stage object detection model based on YOLOv11. The model incorporates three key modules: omni-dimensional dynamic convolution (ODConv), adaptive spatial feature fusion (ASFF), and a multi-point distance IoU (MPDIoU) loss. A class-balanced augmentation strategy is also applied to mitigate category imbalance. We evaluated YOLOv11-OAM on UAV imagery of six fruit tree species—walnut, prune, apricot, pomegranate, saxaul, and cherry. The model achieved a mean Average Precision (mAP@0.5) of 93.1%, an 11.4% improvement over the YOLOv11 baseline. These results demonstrate that YOLOv11-OAM can accurately detect small and overlapping tree crowns in complex orchard environments, offering a reliable solution for precision agriculture and smart forestry applications.