Nitrogen is a major determinant of crop growth, yield formation, and grain quality, making precise nitrogen (N) management essential for sustainable production. This study integrated field canopy spectra acquired with an ASD FieldSpec 4 spectrometer and unmanned aerial vehicle (UAV)-based RGB imagery to estimate leaf nitrogen content in broomcorn millet across four growth stages. Vegetation indices (VIs) and gray-level co-occurrence matrix texture features were derived, and six algorithms—partial least squares (PLS), support vector machine (SVM), random forest (RF), one-dimensional convolutional neural network (CNN1D), one-dimensional residual network (ResNet1D), and one-dimensional U-Net (U-Net)—were evaluated. Among the spectral preprocessing methods, the first-derivative transformation showed the strongest relationship with leaf nitrogen content (r = −0.85). Models based on multi-source feature fusion consistently outperformed those using a single data source. The U-Net model using ASD + RGB + VIs + texture features achieved the highest test-set accuracy (R² = 0.913, RMSE = 1.851, and RPD = 3.382). SHapley Additive exPlanations (SHAP) analysis identified blue-band correlation (B_Correlation) as the most influential individual predictor (mean absolute SHAP value = 0.39), while texture features accounted for 45.170% of the cumulative feature importance. These results demonstrate the potential of multi-source remote sensing and deep learning for rapid, non-destructive assessment of leaf nitrogen content and precision nitrogen management in broomcorn millet.