Early yield estimation is important for improving decision-making and management practices in coffee production systems. However, approaches that use aerial imagery to characterize flowering and relate it to subsequent yield remain limited. This study evaluated the potential of RGB imagery acquired by remotely piloted aircraft (RPA) combined with Deep Learning techniques to classify coffee plants according to three levels of a flowering proxy and to investigate its relationship with yield. A dataset comprising 1593 segmented RGB images was classified into Low, Medium, and High flowering levels using a custom convolutional neural network and three transfer learning architectures: VGG16, ResNet50V2, and MobileNetV2. ResNet50V2 was selected for final evaluation based on its validation performance and achieved 92.12% accuracy, 92.40% macro-precision, 91.81% macro-recall, and 92.06% macro-F1 on the test dataset. The flowering proxy showed a positive association with field-observed yield, with Spearman’s ρ = 0.701 (p < 0.001) and R² = 0.528 for the combined dataset of Arara and Mundo Novo. In consolidated out-of-sample predictions, the model achieved R² = 0.489, MAE = 1.03 kg plant⁻¹, and RMSE = 1.18 kg plant⁻¹. Performance differed between cultivars, with consolidated R² values of 0.567 for Arara and 0.329 for Mundo Novo. Spatial analysis also showed correspondence between the distribution of flowering levels derived from RGB imagery and the subsequent spatial distribution of yield. These findings indicate that RPA-based RGB imagery and Deep Learning can provide spatialized information related to coffee flowering and its productive performance, with potential applications in precision agriculture. Further evaluation across production cycles, cultivars, and environmental conditions is needed to assess the robustness and generalization of the approach.