By analyzing dynamic track geometric irregularity data of high-speed railway (HSR) with ballast track in China, there is a peak of [5, 11] m wavelength for the longitudinal level at the beam joint of simply supported girder bridges. The difference between the absolute value of track longitudinal level at the beam joint and that at the midspan is one to three mm. Under annual temperature cycle, the ballast at the beam joint loosens because of the bridge–track interaction, which results in the longitudinal level at the beam joint gets larger in winter and smaller in summer, that means the longitudinal level is negatively correlated with temperature, and at the same time, the development of the track longitudinal level and Track Quality Index (TQI) with the temperature are given. In addition, the vertical acceleration of axle box and bogie, and wheel–rail vertical force are larger when the train passes the beam joint than when it passes the midspan, which threatens running safety significantly. This article established a three-dimensional vehicle–track–bridge finite element model (FEM), thus the thresholds of track longitudinal level at the beam joint for simply supported girder bridge with ballast track in 250 km/h HSR were proposed, i.e., the maximum of dynamic track longitudinal level is 14 mm, and the maximum of static value is 11 mm. Once one of the two thresholds exceeds the corresponding limit, the train speed should be reduced to 160 km/h or below.