Vacuolar invertase is involved in sugar metabolism and plays a crucial role in plant growth and development, thus regulating seed size. However, information linking vacuolar invertase and seed size in rice is limited. Here we characterized a small grain mutant sg2 (grain size on chromosome 2) that showed a reduced in grain size and 1000-grain weight compared to the wild type. Map-based cloning and genetic complementation showed that OsINV3 is responsible for the observed phenotype. Loss-of-function of OsINV3 resulted in grains of smaller size when compared to the wild type, while overexpression showed increased grain size. We also obtained a T-DNA insertion mutant of OsINV2, which is a homolog of OsINV3 and generated double knockout (KO) mutants of OsINV2 and OsINV3 using CRISPR/Cas9. Genetic data showed that OsINV2, that has no effect on grain size by itself, reduces grain length and width in the absence of OsINV3. Altered sugar content with increased sucrose and decreased hexose levels, as well as changes vacuolar invertase activities and starch constitution in INV3^KO, INV2^KO, INV3^KOINV2^KO mutants indicate that OsINV2 and OsINV3 affect sucrose metabolism in sink organs. In summary, we identified OsINV3 as a positive regulator of grain size in rice, and while OsINV2 has no function on grain size by itself. In the absence of OsINV3, it is possible to detect a role of OsINV2 in the regulation of grain size. Both OsINV3 and OsINV2 are involved in sucrose metabolism, and thus regulate grain size. Our findings increase our understanding of the role of OsINV3 and its homolog, OsINV2, in grain size development and also suggest a potential strategy to improve grain yield in rice.