Segmentation of mining-disturbed land is important for eco-geological environment monitoring. Although existing methods possess strong segmentation capabilities, mining-disturbed land exhibits irregular edges, different spatial sizes, and global texture variability, which lead to difficulties in extracting discriminative features, thereby limiting segmentation accuracy. This study built an RGB-based binary semantic segmentation dataset, covering typical mining-disturbed land in Fujian Province of China. Then a spatial prompt and wavelet Mamba-based multi-scale cross-domain feature fusion network (SWDF-Net) was proposed. (1) Wavelet Mamba-based dual-frequency collaborative enhancement module: high-low frequency information was decoupled by wavelet transform, and collaboratively enhanced by fusion of multiple local details and Mamba-guided global information. It can highlight the high-frequency edge features and low-frequency texture patterns of mining-disturbed land. (2) Cross-domain feature alignment and fusion module: cross-domain statistical calibration and channel conditional modulation were used to narrow spatial–frequency feature distribution gap, which facilitates cross-domain feature alignment and fusion. (3) Spatial prompt-based multi-scale feature weighted fusion module: pixel-level weight maps were generated by spatial prior prompt derived from a spatial decoder and edge-gated branch, which adaptively fuses former multi-scale dual-domain features. The SWDF-Net achieved the best Intersection over Union of 72.22% for mining-disturbed land and performed competitively on the ISPRS Vaihingen and Potsdam datasets.