The task of landslide recognition focuses on extracting the location and extent of landslides over large areas, providing ample data support for subsequent landslide research. This study explores the use of UAV and deep learning technologies to achieve robust landslide recognition in a more rational, simpler, and faster manner. Specifically, the widely successful DeepLabV3+ model was used as a blueprint and a dual-encoder design was introduced to reconstruct a novel semantic segmentation model consisting of Encoder1, Encoder2, Mixer and Decoder modules. This model, named DeepLab for Landslide (DeepLab4LS), considers topographic information as a supplement to DeepLabV3+, and is expected to improve the efficiency of landslide recognition by extracting shape information from relative elevation, slope, and hillshade. Additionally, a novel loss function term—Positive Enhanced loss (PE loss)—was incorporated into the training of DeepLab4LS, significantly enhancing its ability to understand positive samples. DeepLab4LS was then applied to a UAV dataset of Baihetan reservoir, where comparative tests demonstrated its high performance in landslide recognition tasks. We found that DeepLab4LS has a stronger inference capability for landslides with less distinct boundary information, and delineates landslide boundaries more precisely. More specifically, in terms of evaluation metrics, DeepLab4LS achieved a mean intersection over union (mIoU) of 76.0% on the validation set, which is a substantial 5.5 percentage point improvement over DeepLabV3+. Moreover, the study also validated the rationale behind the dual-encoder design and the introduction of PE loss through ablation experiments. Overall, this research presents a robust semantic segmentation model for landslide recognition that considers both optical and topographic semantics of landslides, emulating the recognition pathways of human experts, and is highly suitable for landslide recognition based on UAV datasets.