Frequency modulation continuous wave (FMCW) light detection and ranging (LiDAR) 3D imaging system may suffer from time-varying vibrations which will affect the accuracy of ranging and imaging of a target. The system uses only a single-period FMCW LiDAR signal to measure the range of each spot; however, traditional methods may not work well to compensate for the time-varying vibrations in a single period because they generally assume the vibration velocity is constant. To solve this problem, we propose a time-varying vibration compensation method based on segmented interference. We first derive the impact of time-varying vibrations on the range measurement of the FMCW LiDAR system, in which we divide the time-varying vibration errors into primary errors caused by the vibrations with a constant velocity and quadratic errors. Second, we estimate the coefficients of quadratic vibration errors by using a segmented interference method and build a quadratic compensation filter to eliminate the quadratic vibration errors from the original signals. Finally, we use the symmetrical relations of signals in a triangular FMCW period to estimate the vibration velocity and establish a primary compensation filter to eliminate the primary vibration errors. Numerical tests verify the applicability of this method in eliminating time-varying vibration errors with only a one-period triangular FMCW signal and its superiority over traditional methods.