Joint stiffness of an elastic-joint robot can be changed according to joint stiffness requirements. A series elastic actuator (SEA) can reduce the contact stiffness between the body and the environment or human, which can further ensure interactive operation in a human–machine-compatible environment. However, the introduction of the SEA improves the complexity of the robot dynamics model. In this paper, we propose a control schema based on link-side acceleration measurement to eliminate the overshoot and vibration in the transient process of force control. An extended Kalman filter (EKF) algorithm that fuses photoelectric encoders and accelerometers is first presented based on the link-side acceleration measurement. Following this, based on the external torque estimation, the vibration reduction control algorithm is designed. The simulation model is built, and the algorithm design and simulation of position control and force control are carried out and finally tested on the real robot platform. The effectiveness of the control algorithm is proved. The experimental results show that the dynamic response of the external force estimation is about 2 ms faster than that of the force sensor, and the error between the estimated external torque and the real external torque is within ±0.16 N·m.