Modifiers such as fibers, fillers, natural and synthetic polymer extenders, oxidants and anti-oxidants, and anti-stripping agents are added to produce modified asphalt. However, polymers are the most widely utilized modifiers to enhance the function of asphalt mixtures. The objective of this research was to evaluate the mechanical properties and durability of epoxidized natural rubber (ENR)-modified asphalt mix under short- and long-term aging conditions. The physical and rheological characteristics of the base asphalt and ENR-modified asphalt (ENRMA) were tested. In order to evaluate the mechanical properties and durability of the modified mixtures, the resilient modulus of the ENR–asphalt mixtures under unaged, and short- and long-term aging conditions at various temperatures and frequencies was obtained. Furthermore, the resistance to moisture damage of asphalt mixtures was investigated. The findings showed that the stiffness of the ENR–asphalt mixes increased because of the mutual influence of short- and long-term aging on the mixes. In addition, ENR reduced the susceptibility to moisture damage. The stiffness of the mixes was influenced by the temperature and frequencies. By using mathematical modelling via the multivariable power least squares method, it was found that temperature was the dominant factor among all other factors. The results suggested that the durability of asphalt pavements is improved by using ENR.