Infrared thermography (IRT) is an effective nondestructive testing method for detecting delamination in concrete structures. However, erroneous data interpretation often diminishes its practical utility due to surface irregularities (e.g., color variations) during inspection. These “noisy conditions” alter the temperature distribution of the structure under solar heating cycles, making it challenging to quantify delamination based on the developed thermal contrast (ΔT). This study experimentally investigates the impact of different surface conditions (bare concrete vs. painted surfaces) on ΔT. Artificial delamination at varying depths was simulated and tested under natural environmental conditions, where the maximum ΔT values for shallow delamination were 9.40 °C (bare concrete), 7.35 °C (yellow paint), and 5.15 °C (white paint), respectively. This study measured and analyzed the absorptivity (bare concrete: 0.652, yellow paint: 0.538, and white paint: 0.369), emissivity, and the temperature difference (δT) between non-delaminated areas and air, revealing their correlation with ΔT variation. Based on the results, three typical scenarios are proposed to correlate δT with delamination detection. These findings contribute to a better understanding of the optimal detection window and present a new approach to quantifying delamination at different time windows. The conclusion also discusses the limitations of this study and future research directions.