Rolling bearings are important supporting components of large-scale electromechanical equipment. Once a fault occurs, it will cause economic losses, and serious accidents will affect personal safety. Therefore, research on rolling bearing fault diagnosis technology has important engineering practical significance. Feature extraction with high price density and fault identification are two keys to overcome in the field of fault diagnosis of rolling bearings. This study proposes a feature extraction method based on variational modal decomposition (VMD) and sample entropy and also designs an improved sequence minimization algorithm with optimal parameters to identify the fault. Firstly, a variational modal decomposition system based on vibration signals is designed, and the sample entropy of the components is extracted as the eigenvalue of the signal. Secondly, in order to improve the accuracy of fault diagnosis, the sequence minimum optimization algorithm optimized by the bat algorithm is used as the classifier. Certainly, the traditional bat algorithm (BA) and the sequence minimum optimization algorithm (SMO) are improved, respectively. Therefore, a fault diagnosis algorithm based on IBA-ISMO is obtained. Finally, the experimental verification is designed to prove that the algorithm model has a good state recognition rate for bearings.