Accurate segmentation of lesion region from skin lesion images can provide guidance for medical experts to make an early diagnosis of skin cancer. In this study, we construct Recurrent Attentional Convolutional Networks (O-Net), which exploits the skin lesion’s attention class feature with a recurrent O-shape structure, in an iterative refinement strategy for skin lesion image segmentation. Inspired by the recently proposed attention class feature network, we integrate the attention class feature module into the proposed networks. The O-Net, with recurrent unit to iteratively refine the segmentation result, is designed to extract attention feature information and enable coarse-to-fine feature representation by iteratively integrating attention feature maps into network. Furthermore, O-Net calculates the attentional class feature by extracting attention information from the coarse segmentation result. Two currently popular datasets ISIC-2017 and PH2 are employed to explore the validity of our proposed model. The study provides detailed comparisons of our proposed network, the attention class feature network and Recurrent U-Net. O-Net achieved Dice coefficient by 87.04% on the ISIC-2017 dataset, 92.12% on the PH2 dataset with corresponding Jaccard indices of 80.36% and 86.15%, respectively, on the same dataset, which exhibits competitive performance for skin lesion image segmentation in results. The visual results also shown that more detailed tissues are extracted by O-Net than other methods.