This paper is concerned with the stability of non-monotone traveling waves to a nonlocal dispersion equation with time-delay, a time-delayed integro-differential equation. When the equation is crossing-monostable, the equation and the traveling waves both loss their monotonicity, and the traveling waves are oscillating as the time-delay is big. In this paper, we prove that all non-critical traveling waves (the wave speed is greater than the minimum speed), including those oscillatory waves, are time-exponentially stable, when the initial perturbations around the waves are small. The adopted approach is still the technical weighted-energy method but with a new development. Numerical simulations in different cases are also carried out, which further confirm our theoretical result. Finally, as a corollary of our stability result, we immediately obtain the uniqueness of the traveling waves for the non-monotone integro-differential equation, which was open so far as we know.