Recent advancement of distributed renewable generation has motivated microgrids to trade energy directly with one another, as well as with the utility, in order to minimize their operational costs. Energy trading among microgrids, however, confronts challenges such as reaching a fair trading price, maximizing participants' profit, and satisfying power network constraints. In this paper, we formulate the direct energy trading among multiple microgrids as a generalized Nash bargaining (GNB) problem that involves the distribution network's operational constraints (e.g., power balance equations, voltage limits). We prove that solving the GNB problem maximizes the social welfare and also fairly distributes the revenue among the microgrids based on their market power. To address the nonconvexity of the GNB problem, we propose a two-phase approach. The first phase involves solving the optimal power flow problem in a distributed fashion using the alternative direction method of multipliers to determine the amount of energy trading. The second phase determines the market clearing price and mutual payments of the microgrids. Simulation results on an IEEE 33-bus system with four microgrids show that the proposed framework substantially reduces total network cost by 37.2%. Our results suggest direct trading need be enforced by regulators to maximize the social welfare.

Load aggregators can use demand response programs to motivate residential users toward reducing electricity demand during peak time periods. This paper proposes a demand response algorithm for residential users, while accounting for uncertainties in the load demand and electricity price, users’ privacy concerns, and power flow constraints imposed by the distribution network. To address the uncertainty issues, we develop a deep reinforcement learning (DRL) algorithm using an actor-critic method. We apply federated learning to enable users to determine the neural network parameters in a decentralized fashion without sharing private information (e.g., load demand, users’ potential discomfort due to load scheduling). To tackle the nonconvex power flow constraints, we apply convex relaxation and transform the problem of updating the neural network parameters into a sequence of semidefinite programs (SDPs). Simulations on an IEEE 33-bus test feeder with 32 households show that the proposed demand response algorithm can reduce the peak load by 33% and the expected cost of each user by 13%. Also, we demonstrate the scalability of the proposed algorithm in 330-bus and 1650-bus feeders with real-time pricing scheme. Keywords: demand response, deep reinforcement learning, federated learning, power flow, semidefinite program.

With increasing the presence of co-and tri-generating units, energy hub operators are encouraged to optimally schedule the available energy resources in an economic way. This scheduling needs to be run in an online manner due to the uncertainties in energy prices and demands. In this paper, the real-time scheduling problem of energy hubs is formulated in a dynamic pricing market. The energy hubs interaction is modeled as an exact potential game to optimize each energy hub’s payments to the electricity and gas utilities, as well as the customers’ satisfaction from energy consumption. The potential game approach enables us to study the existence and uniqueness of the Nash equilibrium (NE) and to design an online distributed algorithm to achieve that equilibrium. Simulations results show that the proposed algorithm can increase the energy hubs’ average payoff by 18.8%. Furthermore, energy service companies can improve the technical performance of energy networks by reducing the peak-to-average ratio (PAR) in the electricity and natural gas by 27% and 7%, respectively. When compared with a centralized approach with the objective of social welfare, the proposed algorithm has a significantly lower running time at the cost of lower social welfare.

In the future smart grid, demand response (DR) programs can be deployed to encourage electricity consumers toward scheduling their controllable demands to off-peak hours. Meanwhile, to balance the load and generation, the independent system operator (ISO) requires to motivate the suppliers toward modifying their generation profiles to follow the changes in the load demands. To protect the entities' privacy, the ISO needs to apply an effective pricing scheme to motivate the entities towards modifying their demand and generation profiles. In this paper, we focus on proposing a decentralized DR framework considering the operating constraints of the grid. In our proposed framework, each individual entity responds to the control signals called conjectured prices from the ISO to modify its demand or generation profiles with the locally available information. We formulate the centralized problem of the ISO that jointly minimizes the suppliers' generation cost and the consumers' discomfort cost. We also discuss how the ISO determines the conjectured prices to motivate the entities towards an operating point that coincides with the solution to the centralized problem. The performance of the proposed algorithm is evaluated on a modified IEEE 14-bus in reducing the suppliers' and consumers' cost, as well as transmission lines congestion.

The proliferation of small-scale renewable generators and price-responsive loads make it a challenge for distribution network operators (DNOs) to schedule the generation of the generators and the controllable loads of the load aggregators in real-time. Furthermore, the high computational burden and violation of the entities' privacy (i.e., load aggregators and generators) make a centralized framework impractical. In this paper, we focus on proposing a decentralized energy trading algorithm that can be executed by the entities in a real-time fashion. To address the privacy issue, the DNO provides the entities with proper price signals to motivate them towards a market operating point with maximum profit for the entities and robust renewable generation profile with minimum risk of generation shortage. The performance of the proposed distributed algorithm is evaluated on an IEEE 123-node test feeder. When compared with a benchmark of not performing load management for the aggregators, the proposed algorithm benefits both the load aggregators and generators by increasing their profit by 17.8% and 10.3%, respectively. When compared with a centralized approach, our algorithm converges to the solution of the DNO's centralized problem with a significantly lower running time in about 50 iterations per time slot.

Demand response program with real-time pricing can encourage electricity users toward scheduling their energy usage to off-peak hours. A user needs to schedule the energy usage of his appliances in an online manner since he may not know the energy prices and the demand of his appliances ahead of time. In this paper, we study the users' long-term load scheduling problem and model the changes of the price information and load demand as a Markov decision process, which enables us to capture the interactions among users as a partially observable stochastic game. To make the problem tractable, we approximate the users' optimal scheduling policy by the Markov perfect equilibrium (MPE) of a fully observable stochastic game with incomplete information. We develop an online load scheduling learning (LSL) algorithm based on the actor-critic method to determine the users' MPE policy. When compared with the benchmark of not performing demand response, simulation results show that the LSL algorithm can reduce the expected cost of users and the peak-to-average ratio in the aggregate load by 28% and 13%, respectively. When compared with the short-term scheduling policies, the users with the long-term policies can reduce their expected cost by 17%.