Background and aims: Clinical data suggest that the quality of optical diagnoses of colorectal polyps differs markedly among endoscopists. The aim of this study was to develop a computer program that was able to differentiate neoplastic from non-neoplastic polyps using unmagnified endoscopic pictures. Methods: During colonoscopy procedures polyp photographies were performed using the unmagnified high-definition white light and narrow band image mode. All detected polyps (n = 275) were resected and sent to pathology. Histopathological diagnoses served as the ground truth. Machine learning was used in order to generate a computer-assisted optical biopsy (CAOB) approach. In the test phase pictures were presented to CAOB in order to obtain optical diagnoses. Altogether 788 pictures were available (602 for training the machine learning algorithm and 186 for CAOB testing). All test pictures were also presented to two experts in optical polyp characterization. The primary endpoint of the study was the accuracy of CAOB diagnoses in the test phase. Results: A total of 100 polyps (of these 52% neoplastic) were used in the CAOB test phase. The mean size of test polyps was 4 mm. Accuracy of the CAOB approach was 78.0%. Sensitivity and negative predictive value were 92.3% and 88.2%, respectively. Accuracy obtained by two expert endoscopists was 84.0% and 77.0%. Regarding accuracy of optical diagnoses CAOB predictions did not differ significantly compared to experts (p = .307 and p = 1.000, respectively). Conclusions: CAOB showed good accuracy on the basis of unmagnified endoscopic pictures. Performance of CAOB predictions did not differ significantly from experts’ decisions. The concept of computer assistance for colorectal polyp characterization needs to evolve towards a real-time application prior of being used in a broader set-up.