High-precision segmentation of tobacco leaves is a prerequisite for analysis of phenotypic information. Challenges such as mutual occlusion and fuzzy edges make leaf segmentation difficult. This paper proposes an improved algorithm based on the Mask Region-based Convolutional Neural Networks (MASK RCNN) model and an instance segmentation method based on the SAM model to address these challenges. First, the MASK RCNN model is enhanced by incorporating a feature fusion layer and a hybrid attention mechanism, which improves the segmentation performance. The improved MASK RCNN model achieves an Avg.MIoU metric of approximately 85.10%, which is an improvement of 11.10% over the original algorithm. It also achieves an Avg.MPA metric of about 84.94%, indicating an improvement of 10.84%. Second, the Segment Anything Model (SAM) model is presented for the first time for tobacco leaf segmentation, providing empirical support for its application in the tobacco field. The SAM model demonstrates accurate segmentation of tobacco leaf images at different growth stages, demonstrating its good generality. In conclusion, the proposed methods effectively address the challenges in tobacco leaf segmentation, resulting in improved accuracy and performance. These techniques provide significant technical support for tobacco leaf phenotype research.