In this paper, we address the problem of setting the tap positions of load tap changers (LTCs) for voltage regulation in radial power distribution systems under uncertain load dynamics. The objective is to find a policy to determine the tap positions that only uses measurements of voltage magnitudes and topology information so as to minimize the voltage deviation across the system. We formulate this problem as a Markov decision process (MDP), and propose a batch reinforcement learning (RL) algorithm to solve it. By taking advantage of a linearized power flow model, we propose an effective algorithm to estimate the voltage magnitudes under different tap settings, which allows the RL algorithm to explore the state and action spaces freely offline without impacting the system operation. To circumvent the “curse of dimensionality” resulted from the large state and action spaces, we propose a sequential learning algorithm to learn an action-value function for each LTC, based on which the optimal tap positions can be directly determined. The effectiveness of the proposed algorithm is validated via numerical simulations on the IEEE 13-bus and 123-bus distribution test feeders.

In this paper, we address the problem of jointly determining the energy bid submitted to the wholesale electricity market (WEM) and the energy price charged in the retailed electricity market (REM) for a load serving entity (LSE). The joint bidding and pricing problem is formulated as a Markov decision process (MDP) with continuous state and action spaces, in which the energy bid and the energy price are two actions that share a common objective. We apply the deep deterministic policy gradient (DDPG) algorithm to solve this MDP for the optimal bidding and pricing policies. Yet, the DDPG algorithm typically requires a significant number of state transition samples, which is costly in this application. To this end, we apply neural networks to learn dynamical bid and price response functions from historical data to model the WEM and the collective behavior of the EUCs, respectively. These response functions explicitly capture the inter-temporal correlations of the WEM clearing results and the EUC responses, and can be utilized to generate state transition samples without any cost. More importantly, the response functions also inform the choice of states in the MDP formulation. Numerical simulations illustrated the effectiveness of the proposed methodology.

Maintaining an accurate power system model is crucial for a variety of applications such as electricity markets. In this paper, the cyber security issue induced by critical parameters in electricity market operations is investigated. Critical parameters are a set of model parameters the errors of which cannot be detected due to the lack of local measurements. In this paper, it is first proven that the criticality property of parameters is independent from system operating points, i.e. the maliciously injected errors will always remain undetectable. Then, the cyber attack against critical parameters is formulated as a bilevel programming problem in search of the optimal attack strategy or the worst situation faced by the system operators. An effective algorithm for solving this problem is also provided. Simulation results verify that by injecting undetectable errors into critical model parameters, the locational marginal prices in electricity markets can be significantly distorted, which leads to financial benefits for the attackers.

Battery Energy Storage System (BESS) can be utilized to shave the peak load in power systems and thus defer the need to upgrade the power grid. Based on a rolling load forecasting method, along with the peak load reduction requirements in reality, at the planning level, we propose a BESS capacity planning model for peak and load shaving problem. At the operational level, we consider the optimal control policy towards charging and discharging power with two different optimization objectives: one is to diminish the difference between the peak load and the valley load, the other is to minimize the daily load variance. Particularly, the constraint of charging and discharging cycles, which is an important issue in practice, is taken into consideration. Finally, based on real load data, we provide simulation results that validate the proposed optimization models and control strategies.

The demand response provides an opportunity for load serving entities (LSEs) that operate retail electricity markets (REMs) to strategically purchase energy and provide reserves in wholesale electricity markets (WEMs). This paper concerns with the problem of simultaneously determining the optimal energy bids and reserve offers an LSE submits to the WEM as well as the optimal energy and reserve prices it sets in the REM so as to maximize its profit. To this end, we explicitly model the trilayer market structure that consists of a WEM, a REM, and a set of end user customers, so as to capture the coupling between the bidding problem and the pricing problem. Based on the trilayer market model, we then formulate the joint bidding and pricing problem as a bilevel programming problem and further transform it into a single-level mixed integer linear programming problem, which can be solved efficiently. Numerical studies using the IEEE test cases are presented to illustrate the application of the proposed methodology as well as to reveal several interesting characteristics of the LSE's profit-seeking behavior.