Abiotic stress is one of the key factors harming global agriculture today, seriously affecting the growth and yield of vegetables. Pepper is the most widely grown vegetable in the world, with both high nutritional and economic values. Currently, the increase in global extreme weather events has heightened the frequency of abiotic stresses, such as drought, high and low temperatures, waterlogging, and high salt levels, which impairs pepper growth and development, leading to its reduced yield and quality. In this review, we summarize the research progress on the responses of pepper to abiotic stress in recent years in terms of physiology, biochemistry, molecular level, and mitigation measures. We then explore the existing problems and propose future research directions. This work provides a reference for the cultivation and development of new pepper varieties resistant to abiotic stress.