Although concrete is a noncombustible material, high temperatures such as those experienced during a fire have a negative effect on the mechanical properties. This paper studies the effect of elevated temperatures on the mechanical properties of limestone, quartzite and granite concrete. Samples from three different concrete mixes with limestone, quartzite and granite coarse aggregates were prepared. The test samples were subjected to temperatures ranging from 25 to 650 °C for a duration of 2 h. Mechanical properties of concrete including the compressive and tensile strength, modulus of elasticity, and ultimate strain in compression were obtained. Effects of temperature on resistance to degradation, thermal expansion and phase compositions of the aggregates were investigated. The results indicated that the mechanical properties of concrete are largely affected from elevated temperatures and the type of coarse aggregate used. The compressive and split tensile strength, and modulus of elasticity decreased with increasing temperature, while the ultimate strain in compression increased. Concrete made of granite coarse aggregate showed higher mechanical properties at all temperatures, followed by quartzite and limestone concretes. In addition to decomposition of cement paste, the imparity in thermal expansion behavior between cement paste and aggregates, and degradation and phase decomposition (and/or transition) of aggregates under high temperature were considered as main factors impacting the mechanical properties of concrete. The novelty of this research stems from the fact that three different aggregate types are comparatively evaluated, mechanisms are systemically analyzed, and empirical relationships are established to predict the residual compressive and tensile strength, elastic modulus, and ultimate compressive strain for concretes subjected to high temperatures.