Ultra-high-performance concrete (UHPC) is known for its outstanding strength and durability but is often limited by the high cost of traditional materials, like cement, fine aggregates, and silica fume. This review examines the use of industrial by-products—specifically, iron tailings, steel slag, and desulfurization gypsum—as sustainable alternatives in UHPC mix design. These materials serve as supplementary cementitious components and fine aggregates, helping reduce environmental impacts and production costs. This study highlights the synergistic hydration mechanisms between Portland cement and waste-based materials, leading to improved microstructure and long-term strength. The role of steel fibers in enhancing crack resistance is also discussed. Challenges related to workability, cost, and lack of standardization are addressed, along with opportunities for innovative mix designs, low-carbon binders, and 3D printing. Overall, this paper underscores the potential of industrial by-products to advance sustainable, high-performance UHPC solutions.