Focusing on the problem incurred during particle swarm optimization (PSO) that tends to fall into local optimization when solving Nash equilibrium solutions of games, as well as the problem of slow convergence when solving higher order game pay off matrices, this paper proposes an improved Predator-Prey particle swarm optimization (IPP-PSO) algorithm based on a Predator-Prey particle swarm optimization (PP-PSO) algorithm. First, the convergence of the algorithm is advanced by improving the distribution of the initial predator and prey. By improving the inertia weight of both predator and prey, the problem of “precocity” of the algorithm is improved. By improving the formula used to represent particle velocity, the problems of local optimizations and slowed convergence rates are solved. By increasing pathfinder weight, the diversity of the population is increased, and the global search ability of the algorithm is improved. Then, by solving the Nash equilibrium solution of both a zero-sum game and a non-zero-sum game, the convergence speed and global optimal performance of the original PSO, the PP-PSO and the IPP-PSO are compared. Simulation results demonstrated that the improved Predator-Prey algorithm is convergent and effective. The convergence speed of the IPP-PSO is significantly higher than that of the other two algorithms. In the simulation, the PSO does not converge to the global optimal solution, and PP-PSO approximately converges to the global optimal solution after about 40 iterations, while IPP-PSO approximately converges to the global optimal solution after about 20 iterations. Furthermore, the IPP-PSO is superior to the other two algorithms in terms of global optimization and accuracy.