Fault diagnosis methods based on deep learning have made certain progress in recent years. However, in actual industrial scenarios, there are severe strong background noise and limited computing resources, which poses challenges to the practical application of fault diagnosis models. In response to the above issues, this paper proposes a novel noise-resistant and lightweight fault diagnosis framework with nonlinear timestep degenerative greedy strategy (NTDGS) and dual residual horizontal feature pyramid (DRHFPN) for fault diagnosis in strong noise scenarios. This method takes advantage of the strong fitting ability of deep learning methods to model the agent in reinforcement learning by the ways of parameterization, fully leveraging the advantages of both deep learning and reinforcement learning methods. NTDGS is further developed to adaptively adjust the action sampling strategy of the agent at different training stages, improving the convergence speed of the network. To enhance the noise resistance of the network, DRHFPN is constructed, which can filter out interference noise at the feature map level by fusing local feature details and global semantic information. Furthermore, the feature map weighting attention mechanism (FMWAM) is designed to enhance the weak feature extraction ability of the network through adaptive weighting of the feature maps. Finally, the performance of the proposed method is evaluated in different datasets and strong noise environments. Experiments show that in various fault diagnosis scenarios, the proposed method has better noise resistance, higher fault diagnosis accuracy, and fewer parameters compared to other methods.