Demand side management has been proved to be effective in improving the operating efficiency of microgrids, while posing a severe threat to user privacy. This paper proposes a novel privacy preserving load control scheme for the residential microgrid, in which the microgrid operator manages a multitude of home appliances including electric vehicles (EVs) and air conditioners (ACs). This problem is formulated as a partially observable Markov decision process, since users' privacy information including indoor temperatures associated with ACs and arrival/departure times of EVs cannot be observed by microgrid operator. To address the formulated problem with high-dimensional continuous action space caused by massive controllable appliances, we develop a novel deep reinforcement learning algorithm by introducing credit assignment mechanism. Moreover, we integrate recurrent neural network to accommodate the partial observability of state due to privacy issues. Simulation results demonstrate the superiority and flexibility of the developed algorithm and verify the advantages of the proposed scheme compared with prior privacy preserving load control method.

Decarbonization of the whole energy chain has been recognized as a measure to tackle the global challenge of climate change, and significant progress has already been made on the generation side to integrate renewable energy. However, the demand side is the single largest underlying factor in shaping decarbonization roadmap. Hence, the carbon emission cost should also be shared by the users according to their power consumption. In this paper, a joint electricity-carbon trading framework is designed to reduce the carbon emission through trading and demand response. A delayed carbon emission liability settlement for asynchronous markets is proposed to ameliorate the users' optimal decision from single-point optimization to interval-based optimization. To develop the optimal strategy of trading within the proposed mechanism, an improved proximal policy optimization (PPO) algorithm based on Monte Carlo reward sampling is applied. Simulation studies reveal that, compared with the market without carbon trading and users without delayed settlement, the proposed mechanism has achieved a carbon emission reduction by 40.7% and 12.7% respectively. Simulations also show the algorithm's training efficiency can be significantly improved with the proposed Monte Carlo sampling method.

In demand response programs, load service entity (LSE) can aggregate user as an independent entity to participate in the day ahead energy market, and complete the response target through incentive in the next day. In order to reduce the incentive cost of LSE, a differentiated incentive mechanism that consider the differences in user response flexibility is proposed in this paper. Then, a user response behavior model is established by using the long short-term memory (LSTM) network, with aim of accurately predicting users’ response. Subsequently, an optimization strategy combining particle swarm (PSO) and LSTM is proposed, so that the response target can be accurately completed with low cost. Simulation experiments verified that the cost of LSE is close to the theoretical minimum, and can be reduced by 20% compared with the optimal result under unified incentive mechanism. Moreover, it also verified that the proposed strategy has high response accuracy and good stability.

Energy storage (ES) is playing an increasingly important role in reducing the spatial and temporal power imbalance of supply and demand caused by the uncertainty and periodicity of renewable energy in the microgrid. The utilization efficiency of distributed ES belonging to different entities can be improved through sharing, and considerable flexibility resources can be provided to the microgrid through the coordination of ES sharing and demand response, but its reliability is affected by multiple uncertainties from different sources. In this study, a two-stage ES sharing mechanism is proposed, in which the idle ES capacity is aggregated on the previous day to provide reliable resources for real-time optimization. Then, a two-layer semi-coupled optimization strategy based on a deep deterministic policy gradient is proposed to solve the asynchronous decision problems of day-ahead sharing and intra-day optimization. To deal with the impact of multiple uncertainties, Monte Carlo sampling is applied to ensure that the shared ES capacity is sufficient in any circumstances. Simulation verifies that the local consumption rate of renewable energy is effectively increased by 12.9%, and both microgrid operator and prosumers can improve their revenue through the joint optimization of ES sharing and demand response.

The electrification of transportation has emerged as a notable trend due to advancements in battery technology and the widespread adoption of renewable energy such as photovoltaics (PVs). Many countries have instituted policies aimed at expediting the penetration of electric vehicles (EVs). However, prolonged charging queues and the failure to adequately meet personalized preferences for charging price and time cost have significantly impacted the user experience, thereby impeding the broader adoption of EVs. Moreover, the potential of EVs to use the power from PV panels at charging stations for lower charging tariffs through charging recommendation has not been fully explored. In this paper, we present a charging recommendation method to optimize the drivers’ charging experience, offering three recommendation modes: time priority, price priority and balanced to enhance the compliance level of recommendations. We also consider PV generation in the recommendations, enabling drivers to obtain lower charging tariffs while promoting PV consumption. We formulate the problem as a Markov Decision Process (MDP) and design a customized reinforcement learning (RL) method. Extensive simulations are conducted using the SUMO simulation platform. Results indicate that compared to existing methods, our method promotes the PV consumption ratio by 10.5% and effectively enhances the Quality of Experience (QoE), thereby increasing the recommendation compliance level by 17.4%.