This study proposes a grading algorithm for Orah sorting lines based on machine vision and deep learning. The original ShuffleNet V2 network was modified by replacing the ReLU activation function with the Mish activation function to alleviate the neuron death problem. The ECA attention module was incorporated to enhance the extraction of Orah appearance features, and transfer learning was applied to improve model performance. As a result, the ShuffleNet_wogan model was developed. Based on the operational principles of the sorting line, a time-sequential grading algorithm was designed to improve grading accuracy, along with a multi-sampling diameter algorithm for simultaneous Orah diameter measurement. Experimental results show that the ShuffleNet_wogan model achieved an accuracy of 91.12%, a 3.92% improvement compared to the original ShuffleNet V2 network. The average prediction time for processing 10 input images was 51.44 ms. The sorting line achieved a grading speed of 10 Orahs per second, with an appearance grading accuracy of 92.5% and a diameter measurement compliance rate of 98.3%. The proposed algorithm is characterized by high speed and accuracy, enabling efficient Orah sorting.