Cultivating rice varieties with lower cellulose content in the bran layer has the potential to enhance both the nutritional value and texture of brown rice. This study aims to establish a rapid and accurate method to quantify cellulose content in the bran layer utilizing near-infrared spectroscopy (NIRS), thereby providing a technical foundation for the selection, screening, and breeding of rice germplasm cultivars characterized by a low cellulose content in the bran layer. To ensure the accuracy of the NIR spectroscopic analysis, the potassium dichromate oxidation (PDO) method was improved and then used as a reference method. Using 141 samples of rice bran layer (rice bran without germ), near-infrared diffuse reflectance (NIRdr) spectra, near-infrared diffuse transmittance (NIRdt) spectra, and fusion spectra of NIRdr and NIRdt were used to establish cellulose quantitative analysis models, followed by a comparative evaluation of these models’ predictive performance. Results indicate that the optimized PDO method demonstrates superior precision compared to the original PDO method. Upon examining the established models, their predictive capabilities were ranked in the following order: the fusion model outperforms the NIRdt model, which in turn surpasses the NIRdr model. Of all the fusion models developed, the model exhibiting the highest predictive accuracy utilized fusion spectra (NIRdr-NIRdt (1st der)) derived from preprocessed (first derivative) diffuse reflectance and transmittance spectra. This model achieved an external predictive R²_p of 0.903 and an RMSEP of 0.213%. Using this specific model, the rice mutant O2 was successfully identified, which displayed a cellulose content in the bran layer of 3.28%, representing a 0.86% decrease compared to the wild type (W7). The utilization of NIRS enables quantitative analysis of the cellulose content within the rice bran layer, thereby providing essential technical support for the selection of rice varieties characterized by lower cellulose content in the bran layer.