The effectiveness of water efficiency improvement is restricted by the water rebound effect by which anticipated water resource saving from improved water efficiency may be partly or wholly offset or even surpassed by an increase in water demand. The economy-wide rebound effect of water efficiency improvement in China is poorly understood. This study explored the economy-wide rebound effect of water efficiency improvement in China based on a multi-sectoral computable general equilibrium model. The results suggested that water efficiency improvement could effectively reduce water consumption in producing sectors and benefit economic growth and employment. However, the decrease in water consumption was much lower than the volume of water efficiency improvement, which indicated that the rebound effect partly offset water savings caused by water efficiency improvement. We observed a larger reduction in water consumption in the long run, which indicated a smaller rebound effect and a more significant effect in saving water resources in the long term. Notably, the total rebound effect in the short-run closure was much larger than that in the long-run closure, and the effect from the production side was much smaller. Hence, the economic-wide rebound effect is primarily derived from the incremental water consumption by households, investors, and governments.