This paper aims to develop models to forecast the deterioration of pavement conditions including rutting, roughness, skid-resistance, transverse cracking, and pavement surface distress. A data quality control method was proposed to rebuild the performance data based on the idea of longest increasing or decreasing subsequences. Neural network (NN) was used to develop the five models, and principal component analysis (PCA) was applied to reduce the dimension of traffic variables. The influence of different input variables on the model outputs was discussed respectively by comparing their mean impact values (MIV). Results show that the proposed NN models demonstrated great potential for accurate prediction of pavement conditions, with an average testing R-square of 0.8692. The results of sensitivity analysis revealed that recent pavement conditions may influence the future pavement conditions significantly. Rutting and roughness were sensitive to pavement age and maintenance type. The materials of original pavement asphalt layer were highly relevant to the prediction of pavement roughness, skid-resistance, and pavement surface distress. Moreover, traffic loads obviously affected the pavement skid-resistance and transverse cracking. Pavement and bridge had different effect on surface distress. The material of the base has a remarkable impact on the initiation and development of transverse cracks. Disease treatment in terms of pavement cracking—such as sticking the cracks, excavating and filling the cracks—shows a high MIV in the prediction model of transverse cracking and pavement surface distress.