With extensive pretrained knowledge and high-level general capabilities, large language models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in aspects, such as multitask learning, sample efficiency, and high-level task planning. In this survey, we provide a comprehensive review of the existing literature in LLM-enhanced RL and summarize its characteristics compared with conventional RL methods, aiming to clarify the research scope and directions for future studies. Utilizing the classical agent–environment interaction paradigm, we propose a structured taxonomy to systematically categorize LLMs’ functionalities in RL, including four roles: information processor, reward designer, decision-maker, and generator. For each role, we summarize the methodologies, analyze the specific RL challenges that are mitigated and provide insights into future directions. Finally, the comparative analysis of each role, potential applications, prospective opportunities, and challenges of the LLM-enhanced RL are discussed. By proposing this taxonomy, we aim to provide a framework for researchers to effectively leverage LLMs in the RL field, potentially accelerating RL applications in complex applications, such as robotics, autonomous driving, and energy systems.

Cooperative wind farm control is a complex problem due to wake effect, and it is hard to find the proper model. Reinforcement learning can find the optimal policy in a dynamic environment using “trial and error,” but may damage the machine and cause high cost during the learning process. In order to address this challenge, this article proposes the knowledge-assisted reinforcement learning framework by combining the low-fidelity analytical model with a reinforcement learning framework. Moreover, the knowledge-assisted deep deterministic policy gradient (KA-DDPG) algorithm and three kinds of knowledge-assisted learning methods are proposed based on the framework. The proposed methods are tested in nine different scenarios of WFSim. The simulation results show that the KA-DDPG algorithm can reach the maximum power output and ensure safety during learning. In addition, the learning cost is reduced by accelerating the learning process.

Protecting cybersecurity is a non‐negotiable task for smart grids (SG) and has garnered significant attention in recent years. The application of artificial intelligence, particularly deep learning (DL), holds great promise for enhancing the cybersecurity of SG. Nevertheless, previous surveys and review articles have failed to comprehensively investigate the intersection between DL and SG cybersecurity. To address this gap, this study presents a survey of the latest advancements in DL technology and their relevance to SG cybersecurity. First, the functional mechanisms and scope of application of common DL techniques are explored. Subsequently, SG cyberthreats are categorised into distinct types of cyberattacks that have not been systematically examined in previous surveys. Based on this, a thorough review of the application of DL techniques in addressing each cyberthreat along with recommendations and a generalised framework for enhancing cyberattack detection using DL is offered. Finally, insights are provided into the emerging challenges presented by DL applications in SG cybersecurity that are yet to be widely acknowledged, and potential research avenues are proposed to address or alleviate these challenges.

Applying large language models (LLMs) to modern power systems presents a promising avenue for enhancing decision-making and operational efficiency. However, this action may also incur potential security threats, which have not been fully recognized so far. To this end, this article analyzes potential threats incurred by applying LLMs to power systems, emphasizing the need for urgent research and development of countermeasures.

The real‐time carbon emission estimation of generators helps quantify the carbon emission costs and reduce power system emissions of power generation. Accurate estimation relies on the accuracy of the carbon emission factors (EFs) and power generation measurements. The dynamic carbon emission factor (DEF) of generators was proposed recently as a linear function of the output power. However, there is a significant deviation between the modelled DEFs and the actual measurement EFs, especially when the output power is low. This paper first presents the general definition of the DEF to characterize the emissions and focuses on the unit‐level DEF (UDEF). The piecewise non‐linear UDEF (P‐UDEF) model is then proposed, which can better represent the unit emission characteristics. Then an accurate piecewise linear cost approximation method is proposed considering the segment points and extreme points of both P‐UDEF and generation costs function. Last, the system carbon emissions and costs estimation are estimated by combined economic emission dispatch (CEED), and the reduction potential is evaluated. Case studies on an IEEE 30‐bus system with piecewise linear cost functions show that the proposed P‐UDEF can realize real‐time emission and cost estimation as well as reduce the total system emissions by considering the incomplete combustion cost of generating units.