Due to the existence of flexible joints, the dynamic coupling process of the multi-flexible manipulator system tends to be complicated, and the control of the system tends to be difficult. In this paper, to reduce the influence of joint flexibility on the system, a 3-degrees-of-freedom (3-DOF) dynamic model of flexible joint manipulator is established; and to control the system, the kinematics, trajectory planning and dynamics are integrated. Combine with the proportional-derivative(PD) controller and model predictive controller, the double-layer structure controller is designed. The upper layer of controller adopts the model predictive controller for multi-variable control, and the set value of joint motion of each PD controller in the lower layer is obtained. The simulation results show that the manipulator system based on double-layer structure is easy to control and it has a good stability and a strong applicability.