In this paper, the problem of actuator and sensor faults of a quadrotor unmanned aerial vehicle (QUAV) system is studied. In the system fault model, time delay, nonlinear term, and disturbances of QUAV during the flight are considered. A fault estimation algorithm based on an intermediate observer is proposed. To deal with a single actuator fault, an intermediate variable is introduced, and the intermediate observer is designed for the system to estimate fault. For simultaneous actuator and sensor faults, the system is first augmented, and then two intermediate variables are introduced, and an intermediate observer is designed for the augmented system to estimate the system state, faults, and disturbances. The Lyapunov–Krasovskii functional is used to prove that the estimation error system is uniformly eventually bounded. The simulation results verify the feasibility and effectiveness of the proposed fault estimation method.