The early and accurate recognition of fruit tree pests and diseases is essential for safeguarding fruit yield, quality, and sustainable agricultural production. Conventional manual inspection methods are inadequate for meeting the demands of continuous, objective, and real-time monitoring in large-scale orchards. Following the framework of “model evolution–key challenges–edge-intelligent deployment,” this review systematically summarizes advances in deep learning-based recognition of fruit tree pests and diseases, and compares the effectiveness and limitations of representative methods from the perspectives of data complexity, model generalization and robustness, real-time inference, cross-modal fusion, and trustworthy diagnosis. Existing studies indicate that CNNs, attention mechanisms, Transformers, multimodal fusion, and lightweight networks have promoted the transition of fruit tree pest and disease recognition from image classification to object detection, lesion segmentation, and edge deployment; however, sample scarcity, class imbalance, insufficient cross-domain generalization, black-box decision-making, energy constraints, and long-term robustness remain major bottlenecks for field application. Future research should focus on open orchard environments and develop data-efficient, interpretable, low-power, and continuously updatable edge-intelligent recognition systems, thereby advancing precision agriculture and smart orchards.