With the development of the automotive industry, a large amount of waste rubber is produced every year. The application and development of recycled rubber concrete (RRC) can effectively reduce ‘black pollution’ caused by waste rubber. However, the addition of recycled rubber particles can lead to a decrease in the compressive behavior of concrete. Previous research has demonstrated that by preventing crack growth, fiber addition can increase the strength and ductility of concrete. In this work, a total of 28 RRC mixes are designed, and the compressive behavior of RRC reinforced by steel fibers (SFs) and glass fibers (GFs) is investigated. The workability of fresh RRC can be negatively impacted by an increase in both fiber contents, with the GF content having a more notable effect. With the addition of fibers, the maximum increase rates for the compressive strength, elastic modulus, strain at peak stress, and compressive toughness were 27%, 8%, 45%, and 152%, respectively. A constitutive model is concurrently put forward to forecast the stress–strain curves of RRC with various fiber contents. These findings indicate that the maximum improvement in compressive behavior is achieved when the GF content was 0.4% and the SF content was 1.2%. The proposed constitutive model can be used to predict the stress–strain curve of hybrid fiber-reinforced recycled rubber concrete (HFRRRC).