Shale reservoir has an initial unloading effect during the natural uplift and erosion process, which causes the shale brittleness to change, affecting the design of the fracturing scheme. To consider this, the axial compression loading and confining pressure unloading experiment of shale is carried out, and then the influence of unloading rate on the mechanical parameters, failure characteristics, and the brittleness of rock are analyzed. What is more, a new evaluation method of brittleness characteristics that take the unloading effect into consideration is proposed. The conclusions are as follows: (1) The unloading rate has a weakening effect on the mechanical parameters, such as the destructive confining pressure and the residual strength of the samples. (2) The failure characteristics of shale specimens are a single shear failure in an oblique section under low unloading rate, and multiple shear zones accompanied with bedding fracture under high unloading rate. (3) The brittleness of shale samples is well verified by the brittleness index $B_{d1}$ and $B_{d2}$ during the loading path; nevertheless, it has shortage at the unloading path. This paper proposes a new brittleness evaluation method which can consider the influence of the different unloading rates and unloading points. Furthermore, there is a nice characterization between the brittleness damage and this method.