This paper investigates the impact of green credit policy (GCP) on the green technological innovation capacity of heavily polluting enterprises (HPEs) from the perspectives of external financing and internal concerns. Using data from companies in China’s A-share market from 2008 to 2021, we study the shocks of Green Credit Guidelines to the green technological innovation in HPEs based on the difference-in-differences (DID) model. The baseline regression result reveals that the GCP significantly motivates HPEs to engage in green technological innovation. Moreover, the efficiency of capital utilization has an adverse moderating effect on the impact of GCP, while commercial credit financing has a positive moderating effect. Mechanism analysis shows that the GCP stimulates green technological innovation in HPEs by reinforcing short-term loan dependence and strengthening executive green awareness. In a further study, the policy effects are heterogeneous for enterprises with different characteristics. Regionally, green credit policy affects enterprises in China’s eastern region more strongly. The effect is also more pronounced for Chinese domestic enterprises and those with low financial background heterogeneity within top management teams. Overall, the findings in this study have important implications for policymakers in implementing green finance policies.