Soil moisture is an important factor determining yield. With the increasing demand for agricultural irrigation water resources, evaluating soil moisture in advance to create a reasonable irrigation schedule would help improve water resource utilization. This paper established a continuous system for collecting meteorological information and soil moisture data from a litchi orchard. With the acquired data, a time series model called Deep Long Short-Term Memory (Deep-LSTM) is proposed in this paper. The Deep-LSTM model has five layers with the fused time series data to predict the soil moisture of a litchi orchard in four different growth seasons. To optimize the data quality of the soil moisture sensor, the Symlet wavelet denoising algorithm was applied in the data preprocessing section. The threshold of the wavelets was determined based on the unbiased risk estimation method to obtain better sensor data that would help with the model learning. The results showed that the root mean square error (RMSE) values of the Deep-LSTM model were 0.36, 0.52, 0.32, and 0.48%, and the mean absolute percentage error (MAPE) values were 2.12, 2.35, 1.35, and 3.13%, respectively, in flowering, fruiting, autumn shoots, and flower bud differentiation stages. The determination coefficients (R²) were 0.94, 0.95, 0.93, and 0.94, respectively, in the four different stages. The results indicate that the proposed model was effective at predicting time series soil moisture data from a litchi orchard. This research was meaningful with regards to acquiring the soil moisture characteristics in advance and thereby providing a valuable reference for the litchi orchard’s irrigation schedule.