Evolutionary Reinforcement Learning (ERL), which integrates Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for optimization, has demonstrated remarkable performance advancements. By fusing both approaches, ERL has emerged as a promising research direction. This survey offers a comprehensive overview of the diverse research branches in ERL. Specifically, we systematically summarize recent advancements in related algorithms and identify three primary research directions: EA-assisted Optimization of RL, RL-assisted Optimization of EA, and synergistic optimization of EA and RL. Following that, we conduct an in-depth analysis of each research direction, organizing multiple research branches. We elucidate the problems that each branch aims to tackle and how the integration of EAs and RL addresses these challenges. In conclusion, we discuss potential challenges and prospective future research directions across various research directions. To facilitate researchers in delving into ERL, we organize the algorithms and codes involved on https://github.com/yeshenpy/Awesome-Evolutionary-Reinforcement-Learning

During the flood discharge in large-scale hydraulic engineering projects, intense flow-induced vibrations may occur in hydraulic gates, gate piers, spillway guide walls, etc. Furthermore, the vibration mechanism is complicated. For the spillway guide wall, existing studies on the vibration mechanism usually focus on the vibrations caused by flow excitations, without considering the influence of dam vibration. According to prototype tests, the vibrations of the spillway guide wall and the dam show synchronization. Thus, this paper presents a new vibration mechanism of associated-forced coupled vibration (AFCV) for the spillway guide wall to investigate the dynamic responses and reveal coupled vibrational properties and vibrational correlations. Different from conventional flow-induced vibration theory, this paper considers the spillway guide wall as a lightweight accessory structure connected to a large-scale primary structure. A corresponding simplified theoretical model for the AFCV system is established, with theoretical derivations given. Then, several vibrational signals measured in different structures in prototype tests are handled by the cross-wavelet transform (XWS) to reveal the vibrational correlation between the spillway guide wall and the dam. Afterwards, mutual analyses of numeral simulation, theoretical derivation, and prototype data are employed to clarify the vibration mechanism of a spillway guide wall. The proposed mechanism can give more reasonable and accurate results regarding the dynamic response and amplitude coefficient of the guide wall. Moreover, by changing the parameters in the theoretical model through practical measures, the proposed vibration mechanism can provide benefits to vibration control and structural design.

Massively Multiplayer Online Games (MMOs) feature intricate virtual economies that permeate various in-game activities. However, the balancing act between profitability and equality in MMO economic design proves to be a persistent conundrum, especially in nascent business models like Pay-to-Win (P2W). Conventional efforts are curtailed by two primary constraints: the inability to verify and the provision of suboptimal solutions. In light of these predicaments, this paper delves into MMO economies and explores the promising potential of integrating emerging AI methodologies into economic design. Specifically, we introduce a novel hierarchical Reinforcement Learning (RL) solution for achieving Pareto optimality between profitability and equality in P2W economies. Leveraging our substantial industrial acumen and expertise, we establish an economic simulation environment that facilitates authentic and realistic assessments of MMO economic evolution. Building upon this foundation, we reconceptualize the P2W economic design process within the paradigm of a Markov Decision Process (MDP) and tackle it as a standard RL problem. Comprehensive evaluations corroborate that our solution demonstrates consistent personality specialization in economic simulations akin to real-world MMOs and significantly outperforms other baselines in economic design. Further discussions highlight its superiority in both frontier research and practical applications within the game industry.

In this paper, we focus on the decision-making challenge in autonomous driving, a central and intricate problem influencing the safety and practicality of autonomous vehicles. We propose an innovative hierarchical imitation learning framework that effectively alleviates the complexity of learning in autonomous driving decision-making problems by decoupling decision-making tasks into sub-problems. Specifically, the decision-making process is divided into two levels of sub-problems: the upper level directs the vehicle's lane selection and qualitative speed management, while the lower level implements precise control of the driving speed and direction. We harness Transformer-based models for solving each sub-problem, enabling overall hierarchical framework to comprehend and navigate diverse and various road conditions, ultimately resulting in improved decision-making. Through an evaluation in several typical driving scenarios within the SMARTS autonomous driving simulation environment, our proposed hierarchical decision-making framework significantly outperforms end-to-end reinforcement learning algorithms and behavior cloning algorithm, achieving an average pass rate of over 90%. Our framework's effectiveness is substantiated by its commendable achievements at the NeurIPS 2022 Driving SMARTS competition, where it secures dual track championships.

Ground vibration induced by high dam flood discharge has been reported to cause severe structure safety threats and environmental issues. In this paper, an evaluation and reduction approach for ground vibration using systematic and comprehensive studies is proposed. Based on the results of hydraulic physical model tests, the hydrodynamic excitation on the stilling basin plate (SBP) is analytically expressed as spatially variable harmonic loads by the three-dimensional least squares method. Afterwards, a theoretical model for the SBP–foundation coupled system is established and the vibration of SBP subjected to analytical hydrodynamic load input is calculated. The ground vibration is further evaluated through the numerical simulation regarding the SBP vibration as the input load. According to the prototype test result, it is found that the variation trend of evaluation results under different working conditions is consistent with the actual situation, which indicates the effectiveness of this evaluation approach. Furthermore, the sensitivity analysis of SBP physical dimensions to ground vibration is conducted, and an optimized design for SBP is presented as a result. The verification results indicate that ground vibration can be significantly reduced by applying SBP optimization.

Electronic proceedings of IJCAI 2022