High-resolution remote sensing images (HRRSIs) play an important role in large area and real-time earth observation tasks. However, HRRSIs typically comprise heterogeneous objects of various sizes and complex boundary lines, which pose challenges to HRRSI segmentation. Despite the fact that deep convolutional neural networks dramatically boosted the accuracy, several limitations exist in standard models. Existing methods, mainly concatenate multiscale information to extract the various sizes of objects. However, these methods ignore differentiating information, making it difficult to take advantage of them and completely extract small objects. In addition, there have remained some difficulties in extracting boundary information with positions of uncertainty in previous works. In this article, we propose a novel multiscale feature weighted-aggregating and boundary enhancement network (MFBE-Net) for the segmentation of HRRSIs. ResNet-50, possessing a strong ability to extract features, is employed as the backbone. To fully utilize the information that was extracted, we propose a multiscale feature weighted-aggregating module, which aims to weight-integrate deep features, shallow features, and global information. The boundary enhancement module is designed to solve the blurry boundary information problems and locate its positions. Coordinate attention is also applied in the framework to coherently label size-varied ground objects from different categories and reduce information redundancy. Meanwhile, a mixed loss function is used to supervise the network training process. Finally, MFBE-Net was verified on two public HRRSI datasets, and the experimental results show that the proposed framework outperformed other existing mainstream deep learning methods and could further improve the accuracy of HRRSI segmentation.