Rice, the staple food of over half of the world’s population, holds the peculiarity of being the most widely cultivated crop in the world. About 67% of world's population is associated with agriculture sector, therefore it is very important to grow desired kind of seeds. Unknowingly sowing the wrong variety of seeds may lead to colossal waste of effort and money. Also the phenomenon of rice grain adulteration is continuously increasing in high-quality rice and could break the trust of rice importers and exporters. Therefore, the most important objective for quality analysis is to devise an effective system for classifying rice grain varieties. Human judgments in recognizing objects are uneven, subjective, and delayed, therefore traditional rice grain classification methods are expensive and tiresome. Computer vision provides a good alternative that constitutes a current, advanced technology for image processing and data evaluation that holds tremendous promise and potential. In this research study, five varieties of rice seeds including Jehlum, Sr-1, Mushkibudji, Sr-2, and Sr-4 were gathered from local grain and were used for research analysis. Then, in comparison to existing Machine Learning (ML) approaches, we designed a computer vision system "RiceNet" contingent upon Deep Convolutional Neural Network (DCNN) framework for ameliorating the accuracy of identifying five unique groups of rice grain varieties. Deep Learning (DL)-based pre-trained architectures including InceptionV3 and InceptionResNetV2 models were also adopted for classifying five specific groups of rice species. To optimize model parameters and alleviate back-propagation error during training, we utilized Adam optimizer with a learning rate (lr) of 0.00003 to fine-tune our pre-trained InceptionV3 and ResNetInceptionV2 models. We also compared our proposed RiceNet architecture and pre-trained DCNN models with traditional ML approaches of HOG-SVM, SIFT-SVM, HOG-Logistic Regression(HOG-LR), SIFT-Logistic Regression(SIFT-LR), HOG-KNN, and SIFT-KNN for rice grain classification. With these experimentations at hand, we spectacled that our proposed model "RiceNet" outperformed other approaches in similar computer vision tasks. The prediction accuracy outcome for the test dataset by HOG-SVM, SIFT-SVM, HOG-LR, SIFT-LR, HOG-KNN, and SIFT-KNN models were 66.0%, 65.33%, 62.67%, 65.0%, 54.0%, and 52.0% respectively. RiceNet, InceptionV3 and ResNetInceptionV2 have the best prediction accuracy of 94%, 84% and 81.333%. The remarkably high success rate of DCNN models makes them highly valuable and can be extended to endorse an integrated grain identification system that can operate in real-world situations.