In the past, most researchers have explained the three-stage creep behavior of asphalt mixture in detail. Still, there is no reasonable model to describe the creep of the consolidation effect. To accurately describe the consolidation effect of an asphalt mixture during the viscoelastic deformation process, a modified time hardening model was established by using the Malthus model and the Logistic function to change its creep strain and creep compliance. According to the characteristics of asphalt mixture creep, a single penetration creep test was conducted for high-elasticity modified asphalt mixtures at different temperatures (20 °C, 40 °C, 60 °C) and various loading levels (0.55 MPa, 0.70 MPa, 0.85 MPa, 1.00 MPa). The test results showed that the effect of stress on deformation within the normal range of variation was more significant than that of temperature. In addition, the test results were simulated by the modified time hardening model using surface fitting and compared with a time hardening model and a modified Burgers model. A fitting analysis showed that the modified time hardening model more accurately represents the asphalt mixture’s consolidation effect and creep behavior. Therefore, the modified time hardening model can better show the consolidation effect in the creep process.