Heat pumps (HP) are an efficient alternative to non-electric heating systems (NEHS), being a cost-effective mean to support European building sector decarbonization. The paper studies HP and NEHS performance in residential buildings, under different climate conditions and energy tariffs, in six different European countries. Furthermore, a primary energy and environmental analysis is performed to evaluate if the use of HPs is more convenient than NEHS, based on different factors of the electric mix in each country. A specific HP model is developed considering the main physical phenomena occurring along its cycle. Open data from building, climatic and economic sources are used to feed the analysis. Ad hoc primary energy factors and greenhouse gas (GHG) emission coefficients are calculated for the selected countries. The costs and the environmental impact for both heating systems are then compared. The outcomes of the study suggest that, in highly fossil fuels dependent electricity mixes, the use of NEHS represents a more efficient decarbonization approach than HP, in spite of its higher efficiency. Additionally, the actual high price of the electric kWh hampers the use of HP in certain cases.