3D Gaussian Splatting (3D GS) has revolutionized scene representation, efficient photorealistic rendering, and novel view synthesis. However, when applied to complex large-scale real-world scenes, 3D GS faces two major challenges which are geometric errors and rendering artifacts. To address these challenges, we propose FuseGS, a novel cross-modal Gaussian reconstruction framework that seamlessly fuses multi-view images and dense LiDAR point clouds, along with open-vocabulary semantics from large Vision-Language Models (VLMs) to achieve more accurate global geometry and realistic renderings than the existing methods. FuseGS introduces three key innovations. First, to overcome low geometric accuracy in image-only methods, we propose a novel LiDAR-guided Gaussian initialization method to ensure strong geometric supervision. Second, to deal with the difficulty of integrating 2D VLMs semantics into 3D space, we propose a novel category-specific dual-cue refinement method to improve semantics and geometry alignment. Third, to remove rendering artifacts caused by dynamic objects, we introduce a novel semantic-based adaptive density control method to effectively maintain static background reconstruction. To demonstrate the robustness of our framework, we evaluate FuseGS using both self-collected data from handheld and Unmanned Aerial Vehicle (UAV) platforms, as well as public datasets. Compared to the baseline 3D GS method, FuseGS unifies joint optimization of photometric, geometric and static-aware semantics, resulting in 50% improvement in geometric accuracy and 4.8 dB improvement in rendering quality. The method achieves 67.43% mean Intersection over Union (mIoU) without manual annotation. Ultimately, FuseGS has demonstrated excellent potential for Earth observation applications such as smart cities and autonomous driving.