The ground source heat pump heating system is considered as one of the best solutions for the transition towards green heating under the continental climate conditions like Kazakhstan. In this paper, experimental and theoretical investigations were carried out to develop a ground source heat pump-based heating system under the weather conditions in Kazakhstan and to evaluate its thermodynamic performance. The water-to-water heat pump heating system, integrated with a ground source heat exchanger and used refrigerant R134a, was designed to provide hot water to meet the requirements for space heating. The predicted values of the coefficient of performance and the experimental results were found to be in good agreement within 6.2%. The thermodynamic performance of the system was also assessed using various environment-friendly refrigerants, such as R152a, R450A, R513A, R1234yf and R1234ze, as potential replacements for R134a. Although R152a is found to be a good alternative for R134a in terms of coefficient of performance and total equivalent warming impact, its flammability hinders its application. The heating system using refrigerants R450A, R513A, R1234yf and R1234ze shows 2–3% lower coefficient of performance than that of R134a. The highest exergy destruction is found to be attributed to the compressor, followed by the expansion valve, evaporator, and condenser. Considering their low flammability and low environmental impact, R450A, R513A, R1234yf and R1234ze are identified as valuable replacements for R134a.