Ultrasonic non-destructive characterization is an appealing technique for identifying the microstructures of materials in place of destructive testing. However, the existing ultrasonic characterization techniques do not have sufficient long-term gage repeatability and reproducibility (GR&R), since benchmarking data are not updated. In this study, a hierarchical Bayesian regression model was utilized to provide a long-term ultrasonic benchmarking method for microstructure characterization, suitable for analyzing the impacts of experimental setups, human factors, and environmental factors on microstructure characterization. The priori distributions of regression parameters and hyperparameters of the hierarchical model were assumed and the Hamilton Monte Carlo (HMC) algorithm was used to calculate the posterior distributions. Characterizing the nodularity of cast iron was used as an example, and the benchmarking experiments were conducted over a 13-week transition period. The results show that updating a hierarchical model can increase its performance and robustness. The outcome of this study is expected to pave the way for the industrial uptake of ultrasonic microstructure characterization techniques by organizing a gradual transition from destructive sampling inspection to non-destructive one-hundred-percent inspection.