Detection of planted tree seedlings at the individual level is crucial for monitoring forest ecosystems and supporting silvicultural management. The combination of deep learning (DL) object detection algorithms and remote sensing (RS) data from unmanned aerial vehicles (UAVs) offers efficient and cost-effective solutions. However, current methods predominantly rely on unimodal RS data sources, overlooking the multi-source nature of RS data, which may result in an insufficient representation of target features. Moreover, there is a lack of multimodal frameworks tailored explicitly for detecting planted tree seedlings. Consequently, we propose a multimodal object detection framework for this task by integrating texture information from digital orthophoto maps (DOMs) and geometric information from digital surface models (DSMs). We introduce alternate scanning fusion (ASF), a novel multimodal fusion module based on state space models (SSMs). The ASF can achieve global feature fusion while maintaining linear computational complexity. We embed ASF modules into a dual-backbone YOLOv5 object detection framework, enabling feature-level fusion between DOM and DSM for end-to-end detection. To train and evaluate the proposed framework, we establish the planted tree seedling (PTS) dataset. On the PTS dataset, our method achieves an AP50 of 72.6% for detecting planted tree seedlings, significantly outperforming the original YOLOv5 on unimodal data: 63.5% on DOM and 55.9% on DSM. Within the YOLOv5 framework, comparative experiments on both our PTS dataset and the public VEDAI benchmark demonstrate that the ASF surpasses representative fusion methods in multimodal detection accuracy while maintaining relatively low computational cost.