The accurate prediction of pavement performance is essential for transportation administration or management to appropriately allocate resources road maintenance and upkeep. The international roughness index (IRI) is one of the most commonly used pavement performance indicators to reflect the surface roughness. However, the existing research on IRI prediction mainly focuses on using linear regression or traditional machine learning, which cannot take into account the historical effects of IRI caused by climate, traffic, pavement construction and intermittent maintenance. In this work, a long short-term memory (LSTM)-based model, LSTM+MA, is proposed to predict the IRI of pavements using the time-series data extracted from the long-term pavement performance (LTPP) dataset. Effective preprocessing methods and hyperparameter fine-tuning are selected to improve the accuracy of the model. The performance of the LSTM+MA is compared with other state-of-the-art models, including logistic regressor (LR), support vector regressor (SVR), random forest (RF), K-nearest-neighbor regressor (KNR), fully connected neural network (FNN), XGBoost (XGB), recurrent neural network (RNN) and LSTM. The results show that selected preprocessing methods can help the model learn quickly from the data and reach high accuracy in small epochs. Also, it shows that the proposed LSTM+MA model significantly outperforms other models, with an R² of 0.965 and a mean square error (MSE) of 0.030 in the test datasets. Moreover, an overfitting score is proposed in this work to represent the severity degree of the overfitting problem, and it shows that the proposed model does not suffer severely from overfitting.