Martensitic stainless steels (MSSs) have been widely used in the manufacture of turbine blades, surgical instruments, and cutting tools because of their hardness and corrosion resistance. The MSSs are usually tempered at a temperature no higher than 250 °C after quenching to avoid the decline in the hardness, strength, and corrosion resistance of the steels. However, some short-time thermal shocks are inevitable in processes like welding, water grinding, laser marking, etc., in the manufacturing of kitchen knives, all of which may have negative effects on the mechanical properties and corrosion resistance. The effects of these short-time thermal shocks have rarely been studied. In this paper, the martensitic stainless steel 5Cr15MoV (X50CrMoV15 is European Standards) was selected to be tempered at the sensitization temperatures (480 to 600 °C) for a series of times (0.5 to 128 min) after quenching, and the microstructures, hardness, and corrosion resistance of the steel after tempering were investigated. It was shown that the variation in hardness and corrosion resistance of the 5Cr15MoV steel could be divided into four stages over time during tempering at the sensitization temperatures. The hardness of steel was found to increase at first and then decrease with time; accordingly, good corrosion resistance was retained in the initial few minutes of tempering, which then deteriorated fast. The variation in hardness and corrosion resistance of the 5Cr15MoV steel is related to the diffusion of C and Cr atoms at different tempering temperatures. The mechanism of the mechanical properties and corrosion resistance variation caused by the diffusion of C and Cr atoms during tempering at the sensitization temperatures was also discussed.