Unmanned aerial vehicle (UAV) remote sensing provides a flexible and efficient approach to early forest fire monitoring and warning. However, small fire spots and smoke plumes in UAV imagery vary substantially in texture, morphology, and spatial distribution across scales. Interactions between complex forest backgrounds and target features cause information loss during successive feature compression and misaligned spatial and semantic responses during cross-layer fusion. Limited onboard computing and storage further constrain model complexity. To address these challenges, we propose the lightweight feature alignment and multi-scale enhancement network (FAME-Net), which alleviates feature degradation and response mismatches during encoding and fusion while reducing computational redundancy in multi-scale prediction. At the feature encoding stage, the lightweight multi-scale enhanced backbone (LMEB) combines lightweight downsampling with multi-scale context modeling to reduce computational overhead and preserve fine details of small fire spots and weakly textured smoke. Its hierarchical features are fed into the alignment-aware multi-scale feature fusion neck (AMFF-Neck), which coordinates spatial and semantic responses across scales through cross-layer interactions, reducing fusion bias caused by scale discrepancies and background-induced coupling. The resulting features are processed by the lightweight shared convolution detection head (LSCD), which performs scale alignment, shared convolution, and learnable scale compensation for efficient multi-scale prediction. FAME-Net achieves mAP50 values of 71.5% and 65.3% on the integrated forest fire dataset and the Yuxi Wildfire dataset, respectively, with 1.7 M parameters and 3.9 GFLOPs, indicating a favorable accuracy–efficiency trade-off. On an NVIDIA Jetson Orin Nano Super (8 GB), it achieves inference speeds of 36.2 and 31.6 FPS on the two datasets, respectively, further supporting its potential for resource-constrained UAV deployment.