Frontier research primarily focuses on the effect of urban development models on land use efficiency, while ignoring the effect of new-type urban development on the green land use efficiency. Accordingly, this paper employs a super efficiency slacks-based measure (super-SBM) model with undesirable outputs to measure the green land use efficiency based on panel data from 152 prefecture-level cities for the period 2004–2017. We construct a difference-in-differences (DID) model in this paper to test the impact of smart city construction on the green utilization efficiency of urban land and its transmission mechanism. The results showed that: (1) The smart city construction significantly improved the green utilization efficiency of urban land, increasing the general efficiency by 15%. (2) There is significant city-size heterogeneity in the effect of smart city construction on improving green utilization efficiency of urban land. The policy effect is more obvious in mega cities and above than in very-large-sized cities. (3) The city-feature heterogeneity results reveal that, in cities with a higher level of human capital, financial development, and information infrastructure, the effectiveness of smart city construction in improving the green utilization efficiency of urban land are more obvious, and in cities with a higher level of financial development, the effects of the urban policy were more optimal. (4) The smart city construction promotes the green utilization efficiency of urban land through by the information industry development and the regional innovation capabilities.