High voltage spark discharge (HVSD) could generate strong pressure waves that can be combined with a rotary drill bit to improve the penetration rate in unconventional oil and gas drilling. However, there has been little investigation of the effect of electrical conductivity on rock damage and the fragmentation mechanism caused by HVSD. Therefore, we conducted experiments to destroy cement mortar, a rock-like material, in water with five conductivity levels, from 0.5 mS/cm to 20 mS/cm. We measured the discharge parameters, such as breakdown voltage, breakdown delay time, and electrical energy loss, and investigated the damage mechanism from stress waves propagation using X-ray computed tomography. Our study then analyzed the influence of conductivity on the surface damage of the sample by the pore size distribution and the cumulative pore area, as well as studied the dependence of internal damage on conductivity by through-transmission ultrasonic inspection technique. The results indicated that the increase in electrical conductivity decreased the breakdown voltage and breakdown delay time and increased the energy loss, which led to a reduction in the magnitude of the pressure wave and, ultimately, reduced the sample damage. It is worth mentioning that the relationship between the sample damage and electrical conductivity is non-linear, showing a two-stage pattern. The findings suggest that stress waves induced by the pressure waves play a significant role in sample damage where pores and two types of tensile cracks are the main failure features. Compressive stresses close horizontal cracks inside the sample and propagate vertical cracks, forming the tensile cracks-I. Tensile stresses generated at the sample–water interface due to the reflection of stress waves produce the tensile cracks-II. Our study is the first to investigate the relationship between rock damage and electrical conductivity, providing insights to guide the design of drilling tools based on HVSD.