Both epoxy resin (ER) and SBS are considered effective pavement materials for avoiding ruts. However, epoxy resin asphalt suffers from poor low-temperate performance and a high material cost in practical applications. Aiming to tackle these issues, a new type of composite asphalt modifier (ER-SBS) has been fabricated by combining epoxy resin with SBS. This work prepared modified asphalt with different doping amounts using the above composite asphalt modifier (ER-SBS), intending to explore the properties of composite-modified asphalt and the modification mechanism of the modifier. Furthermore, the effects of the composite modifier at different doping amounts on the viscoelastic property of asphalt were explored through rheological tests, and then the prepared composite-modified asphalt was compared with matrix asphalt and SBS-modified asphalt. In addition, the modification mechanism of the composite modifier was investigated by fluorescence microscopy and infrared spectroscopy. The difference in pavement performance between the composite-modified asphalt and SBS-modified asphalt was compared by a rut test and dynamic modulus test. The research results showed that the composite modifier improved the high- and low-temperature performances and viscoelastic property of matrix asphalt. When the doping amount was raised to 9%, the composite-modified asphalt exhibited better a modification effect than SBS-modified asphalt. The rut test results indicated that composite-modified asphalt demonstrated a stronger deformation resistance than SBS-modified asphalt. The dynamic modulus test showed that the composite-modified asphalt has better viscoelastic properties and temperature sensitivity. Fluorescence microscopy suggested that the crosslinking between the composite modifier and asphalt forms a mesh structure which greatly improves its resistance to deformation. From infrared spectroscopy, the composite modifier clearly functions as a physical modifier.