The loading modes and roof lithology have a significant influence on the mechanical properties of coal seams. To reveal the failure modes and energy evolution law of underground coal during the mining process, conventional uniaxial and uniaxial cyclic loading tests were carried out on three types of samples: coal, rock, and coal-rock combinations. The results show that the samples mainly behave with three failure modes (shear slip, tensile splitting, and fracture), and all the coal sections in the coal-rock combinations fail, whereas most rock sections remain intact. The compressive strength of the coal-rock combination is higher than coal and much smaller than rock. Compared with the conventional uniaxial loading condition, both the maximum deformation before failure and Young’s modulus under the cyclic loading condition are greater, and the latter increases quadratically with the cycle index. The energy densities are also calculated, and their variations are analysed in detail. The results show that with increasing cycle index, both the elastic energy stored in the sample and the dissipated energy increase in a quadratic function, and the failure process becomes more intense. This research reveals the failure modes, deformation characteristics, and energy evolution of the coal-rock combination under different loading conditions, which can provide strong support for controlling underground surrounding rocks of the coal face and roadway in coalmines.