The relationship between the various phases of asphalt materials, from asphalt binder to mastic and mixture, has received great attention over the years, with efforts being made to establish linkages among these phases. Many methods for predicting the rheology properties of asphalt mastics from those of asphalt and filler volume fractions exist. However, most prediction methods are based on an empirical formula and on the micromechanical model. Very few research studies focus on the constitutive model. In addition, relatively little research has explored the influence of asphalt–filler interaction on mastic’s rheology properties, which is believed to be an important factor. In this study, the 2S2P1D (two springs, two parabolic elements, and one dashpot) model was applied to link the behavior of asphalt binder, filler volume fraction, asphalt–filler interaction and asphalt mastic. First, the interaction between asphalt and filler was evaluated, and the interaction parameter C of the Palierne model was used as an assessment indicator to calculate the effective filler volume fraction of asphalt mastic. Then, the relation between the 2S2P1D model parameters of asphalt mastic and those of asphalt binder and the effective filler volume fraction was analyzed. Finally, a simple relationship associating the 2S2P1D model parameters $h$, $\log ({\tau }_0)$ of mastic and that of asphalt binder and the effective filler volume fraction was developed. The proposed expression was validated, and the result showed that it was an efficient model for the shear complex modulus prediction of virgin asphalt mastic.