Accurate characterization of the rheological behavior of non-Newtonian fluids is critical in a wide range of industries as it governs process efficiency, safety, and end-product quality. When the rheological behavior of fluid may vary substantially over a relatively short period of time, it is desirable to measure its viscous properties on a more continuous basis than relying on spot measurements made with a viscometer on a few samples. An attractive solution for inline rheological measurements is to measure pressure gradients while circulating fluid at different bulk velocities in a circular pipe. Yet, extracting the rheological model parameters may be challenging as measurement uncertainty may influence the precision of the model fitting. In this paper, we present a method to calibrate the Herschel-Bulkley rheological model to a series of differential pressure measurements made at variable bulk velocities using a combination of physics-based equations and nonlinear optimization. Experimental validation of the method is conducted on non-Newtonian shear-thinning fluid based on aqueous solutions of polymers and the results are compared to those obtained with a scientific rheometer. It is found that using a physics-based method to estimate the parameters contributes to reducing prediction errors, especially at low flow rates. With the tested polymeric fluid, the proportion difference between the estimated Herschel-Bulkley parameters and those obtained using the scientific rheometer are −24% for the yield stress, 0.26% for the consistency index, and 0.30% for the flow behavior index. Finally, the computation requires limited resources, and the algorithm can be implemented on low-power devices such as an embedded single-board computer or a mobile device.