Crop diseases and pests severely threaten global food security. While deep learning has shown promise in controlled settings, real-world field conditions—characterized by complex backgrounds, variable lesion scales, and high inter-class similarity—remain challenging. To overcome limitations in feature representation, scale adaptability, and model efficiency, we propose DPMFNet, a lightweight dual-path network integrating Spatial–Channel Dual-Attention (SCDA) and Multi-Scale Depthwise Separable Convolution (MDSC) modules. SCDA enhances critical regions via dynamic channel–spatial weighting with minimal overhead, while MDSC captures multi-scale contextual information through parallel dilated convolutions. Both are embedded into an improved residual block (AttMDSCBlock) to boost representational power while reducing parameters. A cross-attention mechanism fuses local details and global context from dual pathways, and a lightweight pyramid strategy adaptively integrates features across resolutions. Evaluated on PlantVillage and the AI Challenger 2018 dataset, DPMFNet achieves state-of-the-art accuracy among lightweight models, with only 14.24M parameters and 2.55G FLOPs. It demonstrates superior robustness in complex agricultural environments, balancing performance, efficiency, and deployability.