We propose a simple strategy when training the semi-supervised segmentation model. Traditional semi-supervised segmentation methods use all unlabeled data without any priority. However, it is not reasonable to consider all pseudo labels have the same confidence. Our idea is to divide pseudo labels into two sets, high stable and low stable, to alleviate the negative impact of the wrongly pseudo labeled images. We propose a simple algorithm to calculate the reliability scores of pseudo labels and retraining strategies. The experiments on popular colonoscopy polyp datasets show that our method yields satisfactory results and achieves a Dice Coefficient higher than the result of the traditional semi-supervised method using all unlabeled data about 2.3%. Code is available at https://github.com/sun-asterisk-research/prioritizing_stable_images_ssl