This research focuses on the application of artificial neural networks (ANNs) on parameters extraction of photovoltaic (PV) models. Extracting parameters of the PV models accurately is crucial to control and optimize PV systems. Although many algorithms have been proposed to address this issue, how to extract the parameters of the PV models accurately and reliably is still a great challenge. Neural network algorithm (NNA) is a recently reported metaheuristic algorithm. NNA is inspired by ANNs. Benefiting from the unique structure of ANNs, NNA shows excellent global search ability. However, NNA faces the challenge of slow convergence rate and local optima stagnation in solving complex optimization problems. This article presents an improved NNA, named neural network algorithm with reinforcement learning (RLNNA), for extracting parameters of the PV models. In RLNNA, three strategies, namely modification factor with reinforcement learning (RL), transfer operator with historical population, and feedback operator, are designed to overcome the challenge of NNA. To verify the performance of RLNNA, it is employed to extract the parameters of the three PV models. Experimental results show that RLNNA can extract the parameters of the considered PV models with higher accuracy and stronger stability compared with NNA and the other 12 powerful algorithms, which fully indicates the effectiveness of the improved strategies.