In order to establish an effective early warning system for landslide disasters, accurate landslide displacement prediction is the core. In this paper, a typical step-wise-characterized landslide (Caojiatuo landslide) in the Three Gorges Reservoir (TGR) area is selected, and a displacement prediction model of Extreme Learning Machine with Gray Wolf Optimization (GWO-ELM model) is proposed. By analyzing the monitoring data of landslide displacement, the time series of landslide displacement is decomposed into trend displacement and periodic displacement by using the moving average method. First, the trend displacement is fitted by the cubic polynomial with a robust weighted least square method. Then, combining with the internal evolution rule and the external influencing factors, it is concluded that the main external trigger factors of the periodic displacement are the changes of precipitation and water level in the reservoir area. Gray relational degree (GRG) analysis method is used to screen out the main influencing factors of landslide periodic displacement. With these factors as input items, the GWO-ELM model is used to predict the periodic displacement of the landslide. The outcomes are compared with the nonoptimized ELM model. The results show that, combined with the advantages of the GWO algorithm, such as few adjusting parameters and strong global search ability, the GWO-ELM model can effectively learn the change characteristics of data and has a better and relatively stable prediction accuracy.