To address the problem of large parameters in the ResNet model, this article proposes a rolling bearing fault diagnosis method with gram angle field (GAF) and Ghost-ResNet. For addressing the low fault diagnosis accuracy caused by time dimension information loss in image coding, the raw vibration signal is converted into the Gram difference field image. The Ghost convolution module is employed to replace the standard convolution layer in the ResNet model, and a Ghost-ResNet model is proposed for bearing fault diagnosis for the purpose of reducing the parameter sizes and floating point operations. To improve the performance of the Ghost-ResNet model, the RAdam optimization algorithm and Gramian angular difference fields image coding method is used for model training. Experiments with single fault and compound fault are performed for verifying the superiority of the proposed Ghost-ResNet model. The comparison results show that the GAF image coding method is more powerful in representing fault features than the other methods, and the RAdam optimization algorithm has achieved stable and higher diagnostic results with faster convergence speed. Compared with other diagnostic methods, the Ghost-ResNet model has higher diagnostic accuracy, stronger anti-noise performance, and better generalization ability.