Legume catch crops can enhance soil fertility and promote the N and P supply of the subsequent main crop, especially with low mineral fertilizer use. However, the specific impact of catch crops on arbuscular mycorrhiza formation of the following main crop is unknown. Therefore, the impact of serradella (Ornithopus sativus) vs. bare fallow was tested on mycorrhiza formation, potential soil enzyme activities and plant-available P under subsequently grown barley (Hordeum vulgare) and different fertilization treatments (P-unfertilized—P0; triple superphosphate—TSP; compost—COM; combined—COM + TSP) in a long-term field experiment in northeastern Germany. Catch cropping significantly increased mycorrhiza formation of barley up to 14% compared to bare fallow. The impact of serradella on mycorrhiza formation exceeded that of the fertilization treatment. Serradella led to increased phosphodiesterase activities and decreased ß-glucosidase activities in soil. Plant availability of P was not significantly affected by serradella. These findings provide initial evidence that even serradella as a non-host crop of mycorrhizal fungi can promote the mycorrhiza formation of the subsequent crop and P mobilization in soil. We conclude that the prolonged vegetation cover of arable soils by the use of catch crops can promote P mobilization and transfer from P pools to the following main crops.