In order to strengthen the hovering precision of a small unmanned helicopter with a manipulator for completing a hovering operation, in this study, a double-loop control strategy of position and attitude is designed. In the position loop, a sliding mode controller is proposed. It attains high precision and strong robustness under continuous disturbance. With the manipulator continuously operating during helicopter hovering, the overall center of gravity and the moment of inertia of the system will also change. Therefore, a backstepping sliding mode controller is proposed in the attitude loop for controlling the hovering attitude of the helicopter. The high precision and strong robustness of the control system are proven. The digital simulation presents a position steady-state error of less than 2% under constant disturbance, and the hovering attitude angle fluctuation amplitude of the helicopter is less than 0.05 rad.