Summary: In this paper, we develop a dual‐horizon peridynamics (DH‐PD) formulation that naturally includes varying horizon sizes and completely solves the ‘ghost force’ issue. Therefore, the concept of dual horizon is introduced to consider the unbalanced interactions between the particles with different horizon sizes. The present formulation fulfills both the balances of linear momentum and angular momentum exactly. Neither the ‘partial stress tensor’ nor the ‘slice’ technique is needed to ameliorate the ghost force issue. We will show that the traditional peridynamics can be derived as a special case of the present DH‐PD. All three peridynamic formulations, namely, bond‐based, ordinary state‐based, and non‐ordinary state‐based peridynamics, can be implemented within the DH‐PD framework. Our DH‐PD formulation allows for h‐adaptivity and can be implemented in any existing peridynamics code with minimal changes. A simple adaptive refinement procedure is proposed, reducing the computational cost. Both two‐dimensional and three‐dimensional examples including the Kalthoff–Winkler experiment and plate with branching cracks are tested to demonstrate the capability of the method. Copyright © 2016 John Wiley & Sons, Ltd.