Due to the poor comprehensive performance of traditional rubber powder-modified asphalt (RA) and issues like easy segregation in rubber powder/Styrene–Butadiene–Styrene (SBS) composite-modified asphalt, the application of RA in high-grade highways is limited. This study combined SEBS (Styrene–Ethylene–Butylene–Styrene) and SBS to form SEBS/SBS (SE-S) and investigated the effect of the SE-S system on asphalt performance. The activated desulfurized rubber powder (ARP) was prepared via mechanical thermal oxidation and used to produce ARP/SE-S composite-modified asphalt (ASSA) combined with SE-S. The performance and modification mechanism of ASSA were evaluated through conventional, rheological, and microstructural tests. The results showed that SEBS improved storage stability more effectively than SBS. With SEBS:SBS = 0.4:0.6 and a SE-S content of 2–4%, the modified asphalt exhibited better overall performance. The synergy of ARP and SE-S enhances both low-temperature crack resistance and high-temperature deformation resistance, endowing ASSA with excellent viscoelastic rheological properties. The modification mechanism of ASSA was primarily physical and the changes in chemical bonds were mainly caused by decrosslinking of the rubber powder during ARP preparation. SE-S and ARP fully swelled and crosslinked in the asphalt, exhibiting excellent compatibility and endowing the ASSA with superior stability and performance.