With the increasing adoption of cooperative multi-UAV systems in applications such as cargo delivery and ground reconnaissance, the demand for scalable and efficient path planning methods has grown substantially. However, traditional heuristic algorithms are frequently trapped in local optima, require task-specific manual tuning, and exhibit limited generalization capabilities. Furthermore, their dependence on iterative optimization renders them unsuitable for large-scale real-time applications. To address these challenges, this paper introduces an end-to-end deep reinforcement learning framework that bypasses the reliance on handcrafted heuristic rules. The proposed method leverages an encoder–decoder architecture with multi-head attention (MHA), where the encoder generates embeddings for UAVs and task parameters, while the decoder dynamically selects actions based on contextual embeddings and enforces feasibility through a masking mechanism. The MHA module effectively models global spatial-task dependencies among nodes, enhancing solution quality. Additionally, we integrate a Multi-Start Greedy Rollout Baseline to evaluate diverse trajectories via parallelized greedy searches, thereby reducing policy gradient variance and improving training stability. Experiments demonstrated significant improvements in scalability, particularly in 100-node scenarios, where our method drastically reduced inference time compared to conventional methods, while maintaining a competitive path cost efficiency. A further validation on simulated mission environments and real-world geospatial data (sourced from Google Earth) underscored the robust generalization of the framework. This work advances large-scale UAV mission planning by offering a scalable, adaptive, and computationally efficient solution.