Water-based mud (WBM) faces challenges in high-temperature, high-pressure (HTHP) conditions due to fluid loss and property degradation. Enhancing eco-friendly drilling fluids with optimal rheology is crucial for sustainable, cost-effective, and environmentally safe drilling operations. This study formulated a WBM using green-synthesized zinc oxide (ZnO) nanoparticles (NPs, ~ 45 nm) and tragacanth gum (TG), a biodegradable natural polymer. The synthesized ZnO NPs were comprehensively characterized using energy-dispersive X-ray spectroscopy (EDS), field-emission scanning electron microscopy (FE-SEM), Raman spectroscopy, X-ray photoelectron spectroscopy (XPS), and thermogravimetric analysis (TGA/DTG) to determine their structural, morphological, and chemical properties. Rheological properties, including flow behavior index (n), consistency index (K), plastic viscosity (PV), and yield point (YP), were analyzed at 25, 50, and 75 °C using the Bingham-plastic and Power-law models. The accuracy of the model was validated using Analysis of Variance (ANOVA), which assessed the significance of the results. Additionally, Design Expert software was utilized to optimize the concentrations of TG and ZnO for elevated temperature applications. Moreover, the response surface methodology (RSM) results were evaluated by reporting the R² and accuracy metrics, confirming the strong correlation between predicted and actual values, which demonstrates the model’s robustness. Three optimal samples underwent HTHP filtration tests at 120 °C and 500 psi. The ideal formulation of 750 ppm TG and 0.25 wt% ZnO NPs improved PV by 27.84%, YP by 43.16%, reduced fluid loss by 54.16%, and mud cake thickness by 25%. The optimized sample showed superior performance, with a ‘K’ of 56.12 cp and a ‘n’ of 0.2272, ensuring effectiveness under HTHP conditions. This sustainable formulation reduced environmental contamination risks and drilling fluid consumption while enhancing operational efficiency.