Gradation has an important influence on the mechanical properties of earth and stone materials after compaction, and a reasonable gradation is the key to ensure compaction quality. In this paper, a set of earth and stone materials gradation detection system based on digital image processing was proposed, which was suitable for detection in the construction sites. The system obtained images which were taken from multiple directions of materials transport vehicle, used the thresholding method to segment the images, detected particle contours through edge detection algorithm of Canny to realize the image recognition of particle size, and drew the gradation curve of earth and rock materials finally. It was verified by selecting limestone aggregates of different gradations, and the results showed high accuracy. The system can realize on‐site detection of the earthwork gradation rapidly and accurately at a dam construction site.