The key problem to solving constrained multi-objective optimization problems (CMOPs) is how to achieve a balance between objectives and constraints. Unfortunately, most existing methods for CMOPs still cannot achieve the above balance. To this end, this paper proposes an adaptive constraint relaxation-based evolutionary algorithm (ACREA) for CMOPs. ACREA adaptively relaxes the constraints according to the iteration information of population, whose purpose is to induce infeasible solutions to transform into feasible ones and thus improve the ability to explore the unknown regions. Completely ignoring constraints can cause the population to waste significant resources searching for infeasible solutions, while excessively satisfying constraints can trap the population in local optima. Therefore, balancing constraints and objectives is a crucial approach to improving algorithm performance. By appropriately relaxing the constraints, it induces infeasible solutions to be transformed into feasible ones, thus obtaining more information from infeasible solutions. At the same time, it also establishes an archive for the storage and update of solutions. In the archive update process, a diversity-based ranking is proposed to improve the convergence speed of the algorithm. In the selection process of the mating pool, common density selection metrics are incorporated to enable the algorithm to obtain higher-quality solutions. The experimental results show that the proposed ACREA algorithm not only achieved the best Inverse Generation Distance (IGD) value in 54.6% of the 44 benchmark test problems and the best Hyper Volume (HV) value in 50% of them, but also obtained the best results in seven out of nine real-world problems. Clearly, CP-TSEA outperforms its competitors.