In this paper, we present the design of a fractional-order sliding mode observer (FO-SMO) for actuator fault estimation in a quadrotor unmanned aerial vehicle (QUAV) system. Actuator faults can significantly compromise the stability and performance of QUAV systems; therefore, early detection and compensation are crucial. Sliding mode observers (SMOs) have recently demonstrated their accuracy in estimating faults in QUAV systems under matched uncertainties. However, existing SMOs encounter difficulties associated with chattering and sensitivity to initial conditions and noise. These challenges significantly impact the precision of fault estimation and may even render fault estimation impossible depending on the magnitude of the fault. To address these challenges, we propose a new fractional-order SMO structure based on the Caputo derivative definition. To demonstrate the effectiveness of the proposed FO-SMO in overcoming the limitations associated with classical SMOs, we assess the robustness of the FO-SMO under three distinct scenarios. First, we examined its performance in estimating actuator faults under varying initial conditions. Second, we evaluated its ability to handle significant chattering phenomena during fault estimation. Finally, we analyzed its performance in fault estimation under noisy conditions. For comparison purposes, we assess the performance of both observers using the Normalized Root-Mean-Square Error (NRMSE) criterion. The results demonstrate that our approach enables more accurate actuator fault estimation, particularly in scenarios involving chattering phenomena and noise. In contrast, the performance of classical (non-fractional) SMO suffers significantly under these conditions. We concluded that our FO-SMO is more robust to initial conditions, chattering phenomena, and noise than the classical SMO.