Phosphogypsum (PG) is a major solid by-product generated during wet-process phosphoric acid production. Its stockpiling poses significant environmental risks due to the potential leaching of harmful substances. Therefore, conversion of PG into high-value-added products is urgent. In this work, nano-calcium carbonate (Nano-CaCO₃) was synthesized from PG via a phase-transfer precipitation method. The surface of the as-prepared product was then modified for Hg²⁺ detection. Initially, the nanoparticle surface was functionalized with (3-aminopropyl)triethoxysilane (APTES), followed by grafting with a synthesized rhodamine B-based derivative (RhoB-NCS). The acquired product was denoted as CaCO₃@RhoB-NCS. Physical characterization demonstrated its successful fabrication. The product is an effective sensor for Hg²⁺ detection, exhibiting enhanced fluorescence emission at 600 nm with a detection limit as low as 1.0 × 10⁻⁶ M. Moreover, the CaCO₃@RhoB-NCS was used as a filler for Hg²⁺ test paper. This work demonstrates a promising approach for valorizing solid waste into high-value-added product.