Accurate estimation of wheat leaf nitrogen concentration (LNC) is critical for characterizing ecosystem and plant physiological processes; it can further guide fertilization and other field management operations, and promote the sustainable development of agriculture. In this study, a wheat LNC test method based on multi-source spectral data and a convolutional neural network is proposed. First, interpolation reconstruction was performed on the wheat spectra data collected by different spectral instruments to ensure that the number of spectral channels and spectral range were consistent, and multi-source spectral data were constructed using interpolated, reconstructed imaging spectral data and non-imaging spectral data. Afterwards, the convolutional neural network DshNet and machine learning methods (PLSR, SVR, and RFR) were compared under various scenarios (non-imaging spectral data, imaging spectral data, and multi-source spectral data). Finally, the competitive adaptive reweighted sampling (CARS) and successive projections algorithm (SPA) were used to optimize the LNC detection model. The results show that the model based on DshNet has the highest test accuracy. The CARS method is more suitable for DshNet model optimization than SPA. In the modeling scenario with non-imaging spectral, imaging spectral, and multi-source spectral, the optimized R² is 0.86, 0.82, and 0.82, and the RMSE is 0.29, 0.31, and 0.31, respectively. The LNC visualization results show that DshNet modeling using multi-source spectral data is conducive to the visualization expansion of non-imaging spectral data. Therefore, the method presented in this paper provides new considerations for spectral data from different sources and is helpful for related research on the chemometric task of multi-source spectral data.