To address the degradation of speed regulation performance in the permanent magnet synchronous motor (PMSM) transmission system caused by the gearbox broken tooth fault and disturbances, a fault model-assisted active disturbance rejection control (FMA-ADRC) algorithm is proposed in this paper. First, an electromechanical coupling model of the motor–gearbox transmission system is established based on the dynamic model of the tooth fault. Secondly, a fault model-assisted extended state observer (ESO) in the active disturbance rejection controller is designed, where the periodic torque disturbances caused by the fault are compensated to reduce the estimation burden on the observer. In addition, the observer is nonlinearized to improve the accuracy of tracking disturbances. The observer error of the nonlinear ESO (NESO) is proven to converge to a bounded region within finite time by using the Lyapunov stability proof theory. Finally, the speed regulation performance of the proposed FMA-ADRC controller is verified under different degrees of fault using an experimental platform based on DSP28335 and MATLAB/SIMULINK R2023b. The reliability and superiority of the proposed controller are verified by the experiment results.