In agricultural planting, early plant classification is an indicator of crop health and growth. In order to accurately classify early plants, this paper proposes a classification method combining a dual attention mechanism and multi-scale module. Firstly, the ECA module (Efficient channel attention) is added to enhance the attention of the network to plants and suppress irrelevant background noise; secondly, the MSFN (Multi-scale Feedforward Network) module is embedded to improve the ability to capture complex data features. Finally, CA (Channel attention) is added to further emphasize the extracted features, thus enhancing the discrimination ability and improving the accuracy of the model. The experimental results show an accuracy of 93.20%, precision of 94.53%, recall of 93.27%, and an F1 score of 93.39%. This study can realize the classification of early plants, and effectively distinguish crops from weeds, which is helpful to identify and realize accurate weeding, thus promoting the intelligent and modern process of agricultural production.