In geology and mineralogy, optical microscopic images have become a primary research focus for intelligent mineral recognition due to their low equipment cost, ease of use, and distinct mineral characteristics in imaging. However, due to their close reflectivity or transparency, some minerals are not easily distinguished from other minerals or background. Secondly, the number of background pixels often vastly exceeds the number of pixels for individual mineral particles, and the number of pixels of different mineral particles in the image also varies significantly. These have led to the issue of data imbalance. This imbalance results in lower recognition accuracy for categories with fewer samples. To address these issues, a flexible ensemble learning for semantic segmentation based on multiple optimized Res-UNet models is proposed, introducing dice loss and focal loss functions and incorporating a pre-positioned spatial transformer networks block. Twelve optimized Res-UNet models were used to construct multiple Res-UNet ensemble learnings using heterogeneous ensemble strategies. The results demonstrate that the system integrated with five learners using the weighted voting fusion method (RUEL-5-WV) achieved the best performance with a mean Intersection over Union (mIOU) of 91.65 across all nine categories and an IOU of 84.33 for the transparent mineral (gangue). The results indicate that this ensemble learning scheme outperforms individual optimized Res-UNet models. Compared to the classical Deeplabv3 and PSPNet, this scheme also exhibits significant advantages.