In the field of reinforcement learning, we propose a Correct Proximal Policy Optimization (CPPO) algorithm based on the modified penalty factor β and relative entropy in order to solve the robustness and stationarity of traditional algorithms. Firstly, In the process of reinforcement learning, this paper establishes a strategy evaluation mechanism through the policy distribution function. Secondly, the state space function is quantified by introducing entropy, whereby the approximation policy is used to approximate the real policy distribution, and the kernel function estimation and calculation of relative entropy is used to fit the reward function based on complex problem. Finally, through the comparative analysis on the classic test cases, we demonstrated that our proposed algorithm is effective, has a faster convergence speed and better performance than the traditional PPO algorithm, and the measure of the relative entropy can show the differences. In addition, it can more efficiently use the information of complex environment to learn policies. At the same time, not only can our paper explain the rationality of the policy distribution theory, the proposed framework can also balance between iteration steps, computational complexity and convergence speed, and we also introduced an effective measure of performance using the relative entropy concept.

The Overset Grid method is a promising computational approach for tackling the challenging moving boundary problems in Computational Fluid Dynamics (CFD) simulations. The computational efficiency and accuracy of the method are critically dependent on the effectiveness of the Overset Grid Assembly (OGA) process. However, the OGA process is plagued by unavoidable issues of load imbalance and communication overheads, which adversely impact the parallel efficiency of the method, particularly when dealing with sub-grids in motion. This paper proposes an improved parallel assembly approach as an effective alternative to address these challenges. Specifically, we introduce a Balanced Merging After Decomposition (BMAD) approach, which ensures that each processor possesses a uniform number of cells from each sub-grid after partitioning and a consistent donor search time. In addition, we deploy a fine-grained list to reduce the data transfer domain, thereby minimizing communication redundancy and cost. We validate the efficiency of our approach in the case of a moving Autonomous Underwater Vehicle (AUV). Experimental results in 3 × 106 grid cells indicate that the proposed approach reduces the parallel computational cost of the OGA process by an average of 21.9% and the speedup has increased by 23.9% with 128 processors. Additionally, it demonstrated equally effective and stable performance in tests using 6 × 106 grid cells, especially achieving the highest speedup of 55.0 with 256 processors.