The accurate identification of soybean growth stages is critical for optimizing agricultural interventions, where mistimed treatments can result in yield losses ranging from 2.5% to 40%. Existing deep learning approaches remain limited in scope, targeting isolated developmental phases rather than providing comprehensive phenological coverage. This paper presents a novel object detection architecture DELTA-SoyStage, combining an EfficientNet backbone with a lightweight ChannelMapper neck and a newly proposed DELTA (Denoising Enhanced Lightweight Task Alignment) detection head for soybean growth stage classification. We introduce a dataset of 17,204 labeled RGB images spanning nine growth stages from emergence (VE) through full maturity (R8), collected under controlled greenhouse conditions with diverse imaging angles and lighting variations. DELTA-SoyStage achieves 73.9% average precision with only 24.4 GFLOPs computational cost, demonstrating 4.2× fewer FLOPs than the best-performing baseline (DINO-Swin: 74.7% AP, 102.5 GFLOPs) with only 0.8% accuracy difference. The lightweight DELTA head combined with the efficient ChannelMapper neck requires only 8.3 M parameters—a 43.5% reduction compared to standard architectures—while maintaining competitive accuracy. Extensive ablation studies validate key design choices including task alignment mechanisms, multi-scale feature extraction strategies, and encoder–decoder depth configurations. The proposed model’s computational efficiency makes it suitable for deployment on resource-constrained edge devices in precision agriculture applications, enabling timely decision-making without reliance on cloud infrastructure.