Semantic segmentation of very-high-resolution remote sensing imagery benefits from combining RGB appearance with Digital Surface Model (DSM) height information, especially in urban scenes where spectrally similar objects often differ in elevation. On the ISPRS Vaihingen and Potsdam benchmarks, BATFNet achieves mIoU scores of 84.06% and 85.31%, respectively, outperforming representative RGB–DSM fusion baselines on most land-cover categories. BATFNet is a supervised boundary-aware Transformer fusion network that uses DSM-derived edge priors to guide bidirectional cross-modal attention and decoder refinement. With a dual-branch ResNet-50 backbone for modality-specific feature extraction, the proposed framework effectively integrates RGB and DSM information while recovering fine spatial details. These results show that exploiting DSM-derived structural cues improves boundary delineation and reduces confusion among spectrally similar urban classes.