With the development of the economy and society, energy problems have become a great concern. The heat pump-coupled thermal energy storage (TES) system is a potential form of building heating, which can improve the stability of the grid and promote the consumption of renewable energy. Phase change materials (PCMs) are widely used in the field of building heating, but there are still some problems such as unsatisfactory melting points, low thermal conductivity, phase separation, and supercooling, which limit the application of PCMs in heat pump heating systems. Therefore, it is very important to improve PCMs by a performance improvement method. This work first summarizes the classification, advantages and disadvantages of PCMs, and introduces the connection between PCMs and heat pumps. Then, a detailed summary of PCMs applied in heat pump heating systems is presented, and a comprehensive review of the performance improvement methods for PCMs, which include additives, encapsulation, and eutectic compounds, is discussed. Finally, the existing problems, solutions, and future research directions are proposed. The emphasis of the research is to clarify the influence of PCMs on heat pump performance and the effect of different performance improvement methods on PCMs, and to illustrate the future development direction for PCMs in heat pump heating technologies, including the matching of heat pumps and PCMs, multi-standard decision methods and advanced control strategies.