Ultrasonic testing (UT) is an important method for concrete, and ultrasonic pulse velocity is commonly used to evaluate the quality of concrete materials in existing studies. The ultrasonic pulse velocity of concrete materials is affected by many factors; therefore, it is necessary to establish a quantitative prediction model for the ultrasonic pulse velocity of concrete materials. Based on the multiscale homogenization method, concrete material is divided into different scales of homogenized materials, namely cement paste, mortar, and concrete. Then, a multiscale ultrasonic pulse velocity model is established through a combination of elasticity formulation and the hydration model. At the three scales of cement paste, mortar, and concrete, the elastic parameters and ultrasonic pulse velocity were predicted with the water-to-cement ratio of 0.35, 0.5, and 0.65, respectively. The ultrasonic pulse velocity of concrete with different water-to-cement ratios and different ages were measured in the test and predicted by the model. The results show that the predicted value of ultrasonic pulse velocity is within the error range of ±1.5% of the measured ultrasonic pulse velocity, suggesting that the established prediction model of ultrasonic pulse velocity can reliably predict the velocity change in concrete materials.