In this paper, a highly ductile polyvinyl alcohol fiber engineered cementitious composite (PVA-ECC) was developed by replacing quartz sand (QS) with tuff stone powder (TP) at different replacement ratios of 20%, 40%, 60%, 80%, and 100%. The resulting mechanical properties and drying shrinkage were determined for the developed ECC. Qualitative and quantitative analyses of hydration products, pore structure, and micro-morphology of ECC were conducted by X-ray diffraction, thermogravimetric analysis, Fourier transform infrared spectroscopy, pore size and porosity, and scanning electron microscopic imaging. The influencing mechanism of tuff stone powder content on ECC performance was also studied at a micro level. It was found that with the increase in the replacement ratio of tuff stone powder, the ultimate tensile strain and tensile peak stress of ECC all exhibited an increasing trend, which declined afterward. The variation in compressive and flexural strengths also showed a similar pattern. When the replacement ratio of tuff stone powder was 40%, the ultimate tensile strain, peak tensile stress, flexural strength, and compressive strength were higher than the control group by 15.1%, 4.7%, 16.3%, and 10.7%, respectively. When the content of tuff stone powder did not exceed 80%, it could fill the internal pores of the ECC matrix, which reduced harmful pores. With the increase in tuff stone powder content, calcite content increases gradually while the Ca(OH)₂ amount decreases. It can be seen that tuff stone powder can improve ECC hydration products. However, incorporating tuff stone powder does not produce new hydration products. Incorporating tuff stone powder increased the drying shrinkage of ECC, and the value of drying shrinkage increased with the increase in the replacement ratio of tuff stone powder.