Pathologically, colorectal polyps can be classified as non-adenoma, which do not have a higher risk of developing colorectal cancer over time, and adenoma, which are thought to be precancerous lesions. The accurate identification and precise differentiation of colorectal polyps during colonoscopy play a crucial role in medical treatment. However, manual screening is prone to missed lesions and misinterpretations due to human error and the visual similarities between different types of polyps. Furthermore, the current process is expensive because it entails the pathological assessment of every polyp. With the development of deep learning technology, significant progress in target detection has made deep learning models a more accurate solution for medical imaging analysis. Therefore, to solve the problems of expensive biopsy costs, low identification accuracy, omission, and classification error caused by the similar appearance of non-adenoma and adenoma, a colorectal polyp detection algorithm based on an improved YOLOv5s equipped with a squeeze-and-excitation attention module was developed. The experiment showed that the performance metrics are higher than the original YOLOv5s, which shows the improved YOLOv5s have great potential to assist endoscopists in the decision-making process.