In recent years, explainable artificial intelligence has received increasing attention in the field of bearing fault diagnosis. However, existing interpretability methods, such as Shapley Additive Explanations (SHAP), often rely on the quality of input features. To achieve high diagnostic accuracy, researchers often extract a large number of features from vibration signals across multiple domains, leading to feature redundancy. This redundancy not only increases the computational cost and risk of overfitting in diagnostic models but also dilutes the contributions of core features during interpretability analysis, resulting in biased explanations. To address this challenge, we propose a two-stage interpretable fault diagnosis approach. In the first stage, the Explainable Boosting Machine (EBM) selects core features to reduce redundancy. In the second stage, EBM is enhanced by Random Forest (RF) through residual learning to form the RF-EBM diagnostic model. EBM and SHAP are further used for dual interpretability analysis. Experimental results on public laboratory benchmark datasets demonstrate that the proposed approach achieves good diagnostic performance and outperforms traditional EBM. Overall, the approach reduces redundancy through feature selection, improves diagnostic performance, and makes the decision-making process more transparent, providing a useful methodological reference for trustworthy fault diagnosis in industrial applications.