Dental caries has been considered the heaviest worldwide oral health burden affecting a significant proportion of the population. To prevent dental caries, an appropriate and accurate early detection method is demanded. This proof-of-concept study aims to develop a two-stage computational system that can detect early occlusal caries from smartphone color images of unrestored extracted teeth according to modified International Caries Detection and Assessment System (ICDAS) criteria (3 classes: Code 0; Code 1–2; Code 3–6): in the first stage, carious lesion areas were identified and extracted from sound tooth regions. Then, five characteristic features of these areas were intendedly selected and calculated to be inputted into the classification stage, where five classifiers (Support Vector Machine, Random Forests, K-Nearest Neighbors, Gradient Boosted Tree, Logistic Regression) were evaluated to determine the best one among them. On a set of 587 smartphone images of extracted teeth, our system achieved accuracy, sensitivity, and specificity that were 87.39%, 89.88%, and 68.86% in the detection stage when compared to modified visual and image-based ICDAS criteria. For the classification stage, the Support Vector Machine model was recorded as the best model with accuracy, sensitivity, and specificity at 88.76%, 92.31%, and 85.21%. As the first step in developing the technology, our present findings confirm the feasibility of using smartphone color images to employ Artificial Intelligence algorithms in caries detection. To improve the performance of the proposed system, there is a need for further development in both in vitro and in vivo modeling. Besides that, an applicable system for accurately taking intra-oral images that can capture entire dental arches including the occlusal surfaces of premolars and molars also needs to be developed.