As for the fault diagnosis process of a reciprocating compressor, vibration signals are often non-stationary, nonlinear, and multi-coupled, which makes it difficult to conduct effective fault information extraction. In this paper, a method based on optimized resonance-based sparse signal decomposition (RSSD) and refined composite multiscale dispersion entropy (RCMDE) is proposed. The quality factors in RSSD are optimized by atom search optimization (ASO) primarily, then the optimal quality factors are applied to the RSSD of reciprocating compressor fault signals. The noise interference in the original vibration signal can be effectively distinguished from the low resonance component after decomposition. The genetic algorithm (GA) is employed to optimize the core parameters of RCMDE. Finally, the RCMDE of the low-resonance component is extracted as the eigenvalue for pattern recognition. The experimental study illustrates that the spring failure, valve wear, and normal valve conditions of reciprocating compressors can be effectively distinguished by the proposed method.