Liberalization of the energy system sets the way towards market-based solutions for ancillary service provision. Local reactive power markets are envisioned to achieve more economically and technically efficient reactive power provision to solve voltage control problems in future distribution and transmission grids. However, market-based reactive power procurement is a difficult and yet unsolved problem. This survey provides a comprehensive overview of the characteristics and hardships of reactive power markets. That is followed by a literature overview of reactive power market design, including local markets and markets on system operator level. Further, methods how to analyse reactive power markets are discussed, focusing on market power, game theoretical approaches, Reinforcement Learning, and manipulation of reactive power markets. From this overview, trends and current research gaps are derived and some general research recommendations are given to serve as a guideline for future research in the field of reactive power markets.
Energy market rules should incentivize market participants to behave in a market and grid conform way. However, they can also provide incentives for undesired and unexpected strategies if the market design is flawed. Multi-agent Reinforcement learning (MARL) is a promising new approach to predicting the expected profit-maximizing behavior of energy market participants in simulation. However, reinforcement learning requires many interactions with the system to converge, and the power system environment often consists of extensive computations, e.g., optimal power flow (OPF) calculation for market clearing. To tackle this complexity, we provide a model of the energy market to a basic MARL algorithm in the form of a learned OPF approximation and explicit market rules. The learned OPF surrogate model makes an explicit solving of the OPF completely unnecessary. Our experiments demonstrate that the model additionally reduces training time by about one order of magnitude but at the cost of a slightly worse performance. Potential applications of our method are market design, more realistic modeling of market participants, and analysis of manipulative behavior.
The shift from conventional power plants on transmission level to distributed energy resources in the distribution grids requires procedures to enable efficient and economic reactive power exchange across different voltage levels. In this paper, we propose a multi-level reactive power market that enables reactive power provision from distributed energy resources to higher voltage levels. Each grid operator operates a local reactive power market and offers local reactive power potential, as an intermediary, to superordinate grid operators by passing on its aggregated cost curve and flexibility range. As a result, there is a low requirement of communication between the market participants and each grid operator is free to choose its local market rules as well as the optimization algorithm used. First results from a case study show that our decentralized market approach can realize an economically efficient multi-level reactive power provision that is close to a centrally computed optimal solution, without violating local grid constraints.
In the last years, diverse agent-based concepts for voltage control in distribution grids were presented in literature. All these approaches are developed manually. Up to now, no tailoring approach has been presented to adapt these concepts to different – likewise specific – grid situations. As the effectiveness of voltage control schemes is highly dependent on the specific grid situation, this can result in multiple problems with regard to performance, applicability and expandability of the proposed agent-based control. The ideal case would be a customizable agent system that is automatically tailored to the grid it shall be applied to, considering its specific characteristics. To address that complex task, this paper proposes an approach for the modular composition of voltage control agents by recombining existing concepts from literature to new fully functional agent systems. This approach makes the first move to an optimized and scenario-specific creation of agent-based control systems that are specifically designed and automatically tailored to a given power system.