Aiming at the problems that the semantic representation of information extracted by the shallow layer of the current remote sensing image scene classification network is insufficient, and that the utilization rate of primary visual features decreases with the deepening of the network layers, this paper designs a multi-scale reverse master–slave encoder network (RMSENet). It proposes a reverse cross-scale supplementation strategy for the slave encoder and a reverse cross-scale fusion strategy for the master encoder. This not only reversely supplements the high-level semantic information extracted by the slave encoder to the shallow layer of the master encoder network in a cross-scale manner but also realizes the cross-scale fusion of features at all stages of the master encoder. A multi-frequency coordinate channel attention mechanism is proposed, which captures the inter-channel interactions of input feature maps while embedding spatial position information and rich frequency information. A multi-scale wavelet self-attention mechanism is proposed, which completes lossless downsampling of input feature maps before self-attention operations. Experiments on open-source datasets RSSCN7, SIRI-WHU, and AID show that the classification accuracies of RMSENet reach 97.41%, 97.61%, and 95.9%, respectively. Compared with current mainstream deep learning models, RMSENet has lower network complexity and excellent classification accuracy.