Soil provides ecosystem services, supports human health and habitation, stores carbon and regulates emissions of greenhouse gases. Unprecedented pressures on soil from degradation and urbanization are threatening agro-ecological balances and food security. It is important that we learn more about soil to sustainably manage and preserve it for future generations. To this end, we developed and analyzed a global soil visible–near infrared (vis–NIR) spectral library. It is currently the largest and most diverse database of its kind. We show that the information encoded in the spectra can describe soil composition and be associated to land cover and its global geographic distribution, which acts as a surrogate for global climate variability. We also show the usefulness of the global spectra for predicting soil attributes such as soil organic and inorganic carbon, clay, silt, sand and iron contents, cation exchange capacity, and pH. Using wavelets to treat the spectra, which were recorded in different laboratories using different spectrometers and methods, helped to improve the spectroscopic modelling. We found that modelling a diverse set of spectra with a machine learning algorithm can find the local relationships in the data to produce accurate predictions. The spectroscopic models that we derived are parsimonious and robust, and using them we derived a harmonized global soil attribute dataset, which might serve to facilitate research on soil at the global scale. This spectroscopic approach should help to deal with the shortage of data on soil to better understand it and to meet the growing demand for information to assess and monitor soil at scales ranging from regional to global. We hope that this work might reinvigorate our community’s discussion towards larger, more coordinated collaborations and encourage other contributions. We also hope that use of the database will deepen our understanding of soil so that we might sustainably manage it and push the research outcomes of the soil, earth and environmental sciences towards applications that we have not yet dreamed of

The number of samples used in the calibration data set affects the quality of the generated predictive models using visible, near and shortwave infrared (VIS–NIR–SWIR) spectroscopy for soil attributes. Recently, the convolutional neural network (CNN) has been regarded as a highly accurate model for predicting soil properties on a large database. However, it has not yet been ascertained how large the sample size should be for CNN model to be effective. This paper investigates the effect of the training sample size on the accuracy of deep learning and machine learning models. It aims at providing an estimate of how many calibration samples are needed to improve the model performance of soil properties predictions with CNN as compared to conventional machine learning models. In addition, this paper also looks at a way to interpret the CNN models, which are commonly labelled as a black box. It is hypothesised that the performance of machine learning models will increase with an increasing number of training samples, but it will plateau when it reaches a certain number, while the performance of CNN will keep improving. The performances of two machine learning models (partial least squares regression – PLSR; Cubist) are compared against the CNN model. A VIS–NIR–SWIR spectra library from Brazil, containing 4251 unique sites with averages of two to three samples per depth (a total of 12 044 samples), was divided into calibration (3188 sites) and validation (1063 sites) sets. A subset of the calibration data set was then created to represent a smaller calibration data set ranging from 125, 300, 500, 1000, 1500, 2000, 2500 and 2700 unique sites, which is equivalent to a sample size of approximately 350, 840, 1400, 2800, 4200, 5600, 7000 and 7650. All three models (PLSR, Cubist and CNN) were generated for each sample size of the unique sites for the prediction of five different soil properties, i.e. cation exchange capacity, organic carbon, sand, silt and clay content. These calibration subset sampling processes and modelling were repeated 10 times to provide a better representation of the model performances. Learning curves showed that the accuracy increased with an increasing number of training samples. At a lower number of samples (< 1000), PLSR and Cubist performed better than CNN. The performance of CNN outweighed the PLSR and Cubist model at a sample size of 1500 and 1800, respectively. It can be recommended that deep learning is most efficient for spectra modelling for sample sizes above 2000. The accuracy of the PLSR and Cubist model seems to reach a plateau above sample sizes of 4200 and 5000, respectively, while the accuracy of CNN has not plateaued. A sensitivity analysis of the CNN model demonstrated its ability to determine important wavelengths region that affected the predictions of various soil attributes.