Game theoretic methods and simulations based on reinforcement learning (RL) are often used to analyze electricity market equilibrium. However, the former is limited to a simple market environment with complete information, and difficult to visually reflect the tacit collusion; while the conventional RL algorithm is limited to low-dimensional discrete state and action spaces, and the convergence is unstable. To address the aforementioned problems, this paper adopts deep deterministic policy gradient (DDPG) algorithm to model the bidding strategies of generation companies (GenCos). Simulation experiments, including different settings of GenCo, load and network, demonstrate that the proposed method is more accurate than conventional RL algorithms, and can converge to the Nash equilibrium of complete information even in the incomplete information environment. Moreover, the proposed method can intuitively reflect the different tacit collusion level by quantitatively adjusting GenCos' patience parameter, which can be an effective means to analyze market strategies.

With the emerging concept of sharing-economy, shared electric vehicles (EVs) are playing a more and more important role in future mobility-on-demand traffic system. This paper considers joint charging scheduling, order dispatching, and vehicle rebalancing for large-scale shared EV fleet operator. To maximize the welfare of fleet operator, we model the joint decision making as a partially observable Markov decision process (POMDP) and apply deep reinforcement learning (DRL) combined with binary linear programming (BLP) to develop a near-optimal solution. The neural network is used to evaluate the state value of EVs at different times, locations, and states of charge. Based on the state value, dynamic electricity prices and order information, the online scheduling is modeled as a BLP problem where the decision variables representing whether an EV will 1) take an order, 2) rebalance to a position, or 3) charge. We also propose a constrained rebalancing method to improve the exploration efficiency of training. Moreover, we provide a tabular method with proved convergence as a fallback option to demonstrate the near-optimal characteristics of the proposed approach. Simulation experiments with real-world data from Haikou City verify the effectiveness of the proposed method.

Grid connection of intermittent renewable energy such as wind power and photovoltaic results in challenges of keeping power balance for power system operation. In order to solve this problem, this paper proposed a multi-time scale coordinated scheduling model for the combined system of Wind power-Photovoltaic-Thermal generator-Hydro pumped storage-Battery (WPTHB) by taking advantages of their complementary operation characteristics. The scheduling model is composed of three time scales: the day-ahead scheduling, the 1-hour ahead scheduling and 15-minute ahead scheduling. 1) in the day-ahead scheduling, based on the 24-hour ahead forecast data of Wind-Photovoltaic power and Load demand (WPL), the optimal power outputs of thermal power units are solved from a mix integer linear programming (MILP) model to achieve the minimal operation cost of thermal units. 2) In the 1-hour ahead scheduling, based on power output of thermal units optimized in the day-ahead scheduling and the hourly forecasted WPL, the hydro pumped unit power outputs are optimally dispatched to minimize their operation cost. 3) In the 15-minute ahead scheduling, based on day-ahead optimal power outputs of thermal units and the 1-hour ahead optimal outputs of pumped storage, the battery optimal power generation is obtained from an AC optimal power flow model solved by MATPOWER. Simulation of the New England system has validated that the proposed multi-time scale coordinated scheduling model could fully explore the distinguished power regulation speed and capacities of thermal power units, hydro pumped storages and batteries to effectively track WPL variations and achieve system economic operation simultaneously.

This article proposes a comprehensive technical review on advanced approaches for electric vehicle charging demand management. The survey is divided into two parts. Part I consists of two major sections. In the former section, the applications for participating in electric power markets, including energy market and ancillary service market, are reviewed. The applications for enhancing renewable energy integration as single energy and multienergy sources are reviewed in the latter section.

Internet data centers are booming in the last few years, leading to gigantic amount of energy consumption and greenhouse gas emission. In this paper, a data center microgrid including conventional units, energy storage, and renewable generation is modeled to optimize its electricity bill and carbon footprint. Considering the randomness of electricity price, renewable output and workload distribution, the volatility risk is incorporated to measure the associated operation risk. Then, a day-ahead emission-aware stochastic resource planning scheme is formulated to decide the strategy on power procurement, energy storage operation, batch workload allocation, and unit commitment of conventional units. Simulation results show the effectiveness of the proposed approach.