Rice grain yield prediction with UAV-driven multispectral images are re-emerging interests in precision agriculture, and an optimal sensing time is an important factor. The aims of this study were to (1) predict rice grain yield by using the estimated aboveground biomass (AGB) and leaf area index (LAI) from vegetation indices (VIs) and (2) determine the optimal sensing time in estimating AGB and LAI using VIs for grain yield prediction. An experimental trial was conducted in 2020 and 2021, involving two fertility conditions and five japonica rice cultivars (Aichinokaori, Asahi, Hatsushimo, Nakate Shinsenbon, and Nikomaru). Multi-temporal VIs were used to estimate AGB and LAI throughout the growth period with the extreme gradient boosting model and Gompertz model. The optimum time windows for predicting yield for each cultivar were determined using a single-day linear regression model. The results show that AGB and LAI could be estimated from VIs (R²: 0.56–0.83 and 0.57–0.73), and the optimum time window for UAV flights differed between cultivars, ranging from 4 to 31 days between the tillering stage and the initial heading stage. These findings help researchers to save resources and time for numerous UAV flights to predict rice grain yield.