Mn:0.15Pb(In_1/2Nb_1/2)O₃-0.55Pb(Mg_1/3Nb_2/3)O₃-0.30PbTiO₃ (Mn:PIMNT) pyroelectric chips were prepared by a two-step annealing method. For the two steps, annealing temperatures dependence of microstructure, defects, surface stress, surface roughness, dielectric properties and pyroelectric properties were studied comprehensively. The controlling factors influencing the pyroelectric properties of the Mn:PIMNT crystals were analyzed and the optimum annealing temperature ranges for the two steps were determined: 600–700 °C for the first step and 500–600 °C for the second step. The pyroelectric properties of the thin Mn:PIMNT chips were significantly enhanced by the two-step annealing method via tuning oxygen vacancies and eliminating surface stress. Based on Mn:PIMNT pyroelectric chips annealed at the most favorable conditions (annealed at 600 °C for the first step and 500 °C for the second step), infrared detectors were prepared with specific detectivity D* = 1.63 × 10⁹ cmHz^1/2W⁻¹, nearly three times higher than in commercial LiTaO₃ detectors.