With increasing deployment of distributed energy resources, the energy market which aims for local generation and load profile redistribution is facing the challenge to accommodate various types of participants. To realize social welfare maximization with privacy preserving in a dynamic energy market, this article propose a multiagent reinforcement learning (MARL) method for quotation decision optimization in continuous double auction (CDA)-based peer-to-peer (P2P) energy market. To address the nonstationarity and privacy violation brought by multiagent context, we utilize mean-field approximation to abstract the unauthorized local information of other agents from the public market dynamics. An abstract Q-value function is developed for each agent to infer the neighbor agents' local observation and action through the public clearing results in the dynamic CDA market. Moreover, to avoid sparse reward so as to stabilize the learning process, we propose a dynamic potential-based reward shaping term in the reward. Without altering the learnt optimal policies, the agents can be informed with the additional energy storage state as the reward shaping in each time instants. To validate the effectiveness and economy of our proposed method, simulation studies are conducted on a real-world dataset. Simulation results show that the proposed MARL method produces up to 17% more convergent episodic reward and 67% less energy bills which indicates competitive convergence performance and significant economic benefits.

The coordinated operation of an integrated electricity and heating system (IEHS) can improve overall energy efficiency and provide additional flexibility. However, the inherent uncertainties pertaining to pipeline parameters and ambient temperature in a district heating network (DHN) have not been addressed in the literature, although such uncertainties could affect the quality or even security of IEHS operation. To address this issue, we propose a two-stage robust IEHS scheduling model that considers the uncertainties of the heat load, ambient temperature and heat dissipation coefficients of heating pipelines. The proposed model embedded with bilinear terms is equivalently transformed via the big-M method into an adaptive linear robust optimization problem that can be solved by the column-and-constraint generation (C&CG) algorithm. Case studies are conducted for two test systems. The simulation results show that the proposed method can effectively handle the uncertainties in DHN and improve the robustness of the IEHS.

Due to the growing penetration of renewable energies (REs) in integrated energy system (IES), it is imperative to assess and reduce the negative impacts caused by the uncertain REs. In this paper, an unscented transformation-based mean-standard (UT-MS) deviation model is proposed for the stochastic optimization of cost-risk for IES operation considering wind and solar power correlated. The unscented transformation (UT) sampling method is adopted to characterize the uncertainties of wind and solar power considering the correlated relationship between them. Based on the UT, a mean-standard (MS) deviation model is formulated to depict the trade-off between the cost and risk of stochastic optimization for the IES optimal operation problem. Then the UT-MS model is tackled by a multi-objective group search optimizer with adaptive covariance and Lévy flights embedded with a multiple constraints handling technique (MGSO-ACL-CHT) to ensure the feasibility of Perato-optimal solutions. Furthermore, a decision making method, improve entropy weight (IEW), is developed to select a final operation point from the set of Perato-optimal solutions. In order to verify the feasibility and efficiency of the proposed UT-MS model in dealing with the uncertainties of correlative wind and solar power, simulation studies are conducted on a test IES. Simulation results show that the UT-MS model is capable of handling the uncertainties of correlative wind and solar power within much less samples and less computational burden. Moreover, the MGSO-ACL-CHT and IEW are also demonstrated to be effective in solving the multi-objective UT-MS model of the IES optimal operation problem.