Understanding the thermal effects on rock is critical for the exploration of geothermal resources and the temperature‐driven evolution of Earth. The macroscale mechanical behavior of granite under thermal loading can be complicated due to microcracking and the heterogeneity of minerals. In this study, the thermally induced microcrack propagation of granite was first observed in real time by an ultrahigh‐temperature instrument on an optical microscope. The previous investigation believed that the α–β quartz transition at approximately 573°C leads to a sharp decrease in macroscale mechanical behavior. However, the present experimental results revealed that microcracks initiate at 300°C and coalesce between 400 and 600°C, which is the main reason for the sharp decrease in macroscale mechanical properties of granite. Additionally, an accurate grain‐based model based on the mineralogical morphology of granite samples adopted mechanical parameters of minerals obtained from microscale rock mechanical techniques. It is able to well reproduce the process of microcracking and the strength evolution of granite during heating. The numerical results show that the heterogeneity of thermal expansion and mechanical properties of minerals produce local thermal stress concentrations in granite. The uneven tensile‐compressive stress between feldspar and quartz and the shear stress around biotite result in the initiation and propagation of microcracks. The present work is potentially useful for understanding and predicting the mechanical behavior of granites under thermal loading with different mineral compositions and microstructures.