Several types of fishways are designed to allow the upstream passage of fish during the reproductive migration. Despite being the subject of many studies, improvements are still necessary to ensure an adequate flow for fish passage. This work aims to study the hydraulic parameters of the free-surface flow in the Igarapava HPP fishway, in Brazil, by comparing numerical results with available experimental data. Recent formulations for smoothed particle hydrodynamics were investigated in the numerical simulation using DualSPHysics. Open boundary conditions with buffer layers were applied to simulate inflow and outflow in the model. Three different model settings compared the effectiveness of the density diffusion term and the viscosity coefficient. Spurious oscillations were significantly reduced in the velocity field by using a combination of diffusion term and artificial viscosity coefficient that resulted in a smoothing effect near the walls and fewer oscillations in water level.