The automatic classification of aluminum profile surface defects is of great significance in improving the surface quality of aluminum profiles in practical production. This classification is influenced by the small and unbalanced number of samples and lack of uniformity in the size and spatial distribution of aluminum profile surface defects. It is difficult to achieve high classification accuracy by directly using the current advanced classification algorithms. In this paper, digital image processing methods such as rotation, flipping, contrast, and luminance transformation were used to augment the number of samples and imitate the complex imaging environment in actual practice. A RepVGG with CBAM attention mechanism (RepVGG-CBAM) model was proposed and applied to classify ten types of aluminum profile surface defects. The classification accuracy reached 99.41%, in particular, the proposed method can perfectly classify six types of defects: concave line (cl), exposed bottom (eb), exposed corner bottom (ecb), mixed color (mc), non-conductivity (nc) and orange peel (op), with 100% precision, recall, and F1. Compared with the existing advanced classification algorithms VGG16, VGG19, ResNet34, ResNet50, ShuffleNet_v2, and basic RepVGG, our model is the best in terms of accuracy, macro precision, macro recall and macro F1, and the accuracy was improved by 4.85% over basic RepVGG. Finally, an ablation experiment proved that the classification ability was strongest when the CBAM attention mechanism was added following Stage 1 to Stage 4 of RepVGG. Overall, the method we proposed in this paper has a significant reference value for classifying aluminum profile surface defects.