A greenhouse pot test was conducted to study the impacts of replacing mineral fertilizer with organic fertilizers for one full growing period on soil fertility, tomato yield and quality using soils with different tomato planting history. Four types of fertilization regimes were compared: (1) conventional fertilizer with urea, (2) chicken manure compost, (3) vermicompost, and (4) no fertilizer. The effects on plant growth, yield and fruit quality and soil properties (including microbial biomass carbon and nitrogen, ${\text{NH}}_4^{+}$-N, ${\text{NO}}_3^{-}$-N, soil water-soluble organic carbon, soil pH and electrical conductivity) were investigated in samples collected from the experimental soils at different tomato growth stages. The main results showed that: (1) vermicompost and chicken manure compost more effectively promoted plant growth, including stem diameter and plant height compared with other fertilizer treatments, in all three types of soil; (2) vermicompost improved fruit quality in each type of soil, and increased the sugar/acid ratio, and decreased nitrate concentration in fresh fruit compared with the CK treatment; (3) vermicompost led to greater improvements in fruit yield (74%), vitamin C (47%), and soluble sugar (71%) in soils with no tomato planting history compared with those in soils with long tomato planting history; and (4) vermicompost led to greater improvements in soil quality than chicken manure compost, including higher pH (averaged 7.37 vs. averaged 7.23) and lower soil electrical conductivity (averaged 204.1 vs. averaged 234.6 μS/cm) at the end of experiment in each type of soil. We conclude that vermicompost can be recommended as a fertilizer to improve tomato fruit quality and yield and soil quality, particularly for soils with no tomato planting history.