How to achieve high-precision detection and real-time deployment of the lithology intelligent identification system has significant engineering implications in the geotechnical, geological, water conservation, and mining disciplines. In this study, a lightweight lithology intelligent identification model is proposed to overcome this problem. The MobileNetV2 model is utilized as the basic backbone network to decrease network operation parameters. Furthermore, channel attention and spatial attention methods are incorporated into the model to improve the network’s extraction of complicated and abstract petrographic elements. In addition, based on the findings of network training, computing power performance, test results, and Grad-CAM interpretability analysis and comparison tests with Resnet101, InceptionV3, and MobileNetV2 models. The training accuracy of the proposed model is 98.59 percent, the training duration is 76 min, and the trained model is just 6.38 megabytes in size. The precision (P), recall (R), and harmonic mean (FI-score) were, respectively, 89.62%, 91.38%, and 90.42%. Compared to the three competing models, the model presented in this work strikes a better balance between lithology recognition accuracy and speed, and it gives greater consideration to the rock feature area. Wider and more uniform, strong anti-interference capability, improved robustness and generalization performance of the model, which can be deployed in real-time on the client or edge devices and has some promotion value.