Low-grade faults play an important role in controlling oil and gas accumulations, but their fault throw is small and difficult to identify. Traditional low-grade fault recognition methods are time-consuming and inaccurate. Therefore, this study proposes a combination of a simulated low-grade fault sample set and a self-constructed convolutional neural network to recognize low-grade faults. We used Wu’s method to generate 500 pairs of low-grade fault samples to provide the data for deep learning. By combining the attention mechanism with UNet, an SE-UNet with efficient allocation of limited attention resources was constructed, which can select the features that are more critical to the current task objective from ample feature information, thus improving the expression ability of the network. The network model is applied to real data, and the results show that the SE-UNet model has better generalization ability and can better recognize low-grade and more continuous faults. Compared with the original UNet model, the SE-UNet model is more accurate and has more advantages in recognizing low-grade faults.