The relationship between the strategic choice of internationalization under institutional duality and the performance of emerging market multinational enterprises (EM-MNEs) will become a focal point for scholars, practitioners and policy makers in the future. In order to better understand the determinants of the entry mode choice under institutional duality for EM-MNEs, based on the sample of Chinese manufacturing listed companies ‘going out’ from 2003 to 2015, we investigated the impact of organization institutional inertia and host-country institutional quality on the choice of the international joint venture (IJV) of Chinese manufacturing firms, as well as the effect and potential problems of the IJV choice on the firms’ innovation performance under institutional duality. The main results of our empirical analyses provide evidence that the response to institutional duality means that Chinese firms have the possibility to choose IJV in entry modes ahead of their capability support, and subject to insufficient professional field accumulation, cross-cultural management and joint governance capability, this premature choice of IJV actually hinders their innovation efficiency. Furthermore, the test on the influence of firm heterogeneity found that, when the firm is privately owned or has a lower internationalization breadth, the above negative moderating effect of IJV choice is more significant. Our findings enrich the literature on the relationship between the strategic choice of internationalization and innovation performance of EM-MNEs, and provide inspiring and straightforward empirical evidence.