Macroscopic properties of sandstone are commonly attributed to the degradation of its microstructure during heating treatment processes. However, few previous studies have focused on comprehensive observations on how the microstructure of sandstone changes with temperature. In this study, a kind of sandstone containing quartz, albite, calcite, and laumontite (little), was collected from Linyi (Shandong Province, China) to observe the microstructure degradation changes with temperature by X-ray diffraction (XRD), Scanning electron microscopy (SEM) and thermo-gravimetric analyses (TGA). Firstly, 10 groups of sandstone samples were heated from 25 °C to 900 °C. Then, some core micro-parameters including lattice constant, full width at half maximum (FWHM), micro-strain, dislocation density, TGA curve changes and failure characteristic of the mineral were analyzed comprehensively. Finally, the underlying mechanism causing the microscopic thermal damage at different temperature intervals was also discussed. The results showed that: (1) quartz, the framework component of this sandstone, underwent an α- to β-phase change over the temperature range from 400 °C to 600 °C. This phenomenon caused the lattice constant, micro-strain, dislocation density and TGA curve to decrease sharply during this interval, leading to the microstructure deterioration of sandstone; (2) calcite underwent a decomposition reaction between 600 °C and 800 °C, and resulted in the XRD pattern peak, lattice constant, micro-strain and TGA curve dropping continuously. It destroyed further the internal microstructure of sandstone and produced numerous inter-granular cracks around quartz crystals; (3) further examination found that the decomposition reactions of minerals presented non-synchronized characteristics due to the different sensitivities of minerals to temperature, which led to thermal stress, thermal fracturing of minerals, and thermal reactions happening in different temperature intervals.