With the popularization of electric vehicles (EVs), EV charging demand is becoming an important load in the building. Considering the mobility of EVs from building to building and their uncertain charging demand, it is of great practical interest to control the EV charging process in a microgrid of buildings to optimize the total operation cost while ensuring transmission security between the microgrid and the main grid. We consider this important problem in this article and make the following contributions. First, we formulate this problem as a Markov decision process (MDP) to capture the uncertain supply and EV charging demand in the microgrid of buildings. Besides reducing the total operation cost of buildings, the model also considers the power exchange limitation to ensure transmission security. Second, this model is reformulated within an event-based optimization (EBO) framework to mitigate the impact of large state and action space. A randomized parametric event-based control policy is proposed for the microgrid controller to determine the charge ratio for each building. Then, a selecting-to-charging principle is implemented for each building controller to determine the charge control of each EV in the building. Third, a constrained gradient-based policy optimization method is developed to find the optimal event-based control policy. Furthermore, an adjustment mechanism is also introduced to ensure transmission security during optimization. Numerical experiments are conducted to analyze the structure and the performance of the event-based control policy for EV charging in a microgrid of three buildings.

The electric vehicle (EV) utilizing vehicle-to-grid (V2G) technology can serve as an important mobile storage in a microgrid of building. Since more and more buildings are equipped with charging piles and distributed renewable energy, it can potentially improve the operation efficiency of building microgrid and reduce the impact of EVs to the grid by V2G scheduling considering the uncertain supply and EV charging demand in the building. We consider this important problem in this paper and make the following contributions. First, we formulate this V2G scheduling problem as a constrained Markov decision process (CMDP). The objective function is to improve the overall building energy operation cost while constraining the expected cycling time to ensure the participation enthusiasm of the EV users. Second, a simulation-based primal-dual approach is developed to decompose the original problem into a continuous optimization subproblem on the supply side and a discrete optimization subproblem on the demand side. The demand side optimization can be further decoupled as a distributed single EV scheduling problem where simulation-based regularized rollout method can be applied to improve from existing base policies. Third, the structural property of the problem and the cost improvement of the proposed method are analyzed to speed up the optimization and ensure the solution quality. Numerical experiments based on real building load data and EV data are conducted to demonstrate the performance of this method. Note to Practitioners—With the rapid adoption of EVs in the microgrid of building, there goes the challenge of how to reduce their charging impact to the microgrid. The V2G scheduling between EVs and building microgrid can provide a promising way to reduce the impact and improve the building operation efficiency. However, it suffers from the uncertain EV charging demand and the conflict between synthesized coordination and ensuring the participation enthusiasm of drivers. In this paper, we propose a constrained V2G scheduling model to address this issue. After formulating the problem as a constrained Markov decision process, a simulation-based primal-dual approach is developed to decompose the problem where the derived control policy in the supply side can be deployed in the building energy controller and the derived control policy in the demand side can be deployed in the controller of each charging pile. Case studies show the performance of the proposed method and its win-win property for the building and EV users.

The applications of unmanned swarms have become increasingly widespread, gradually transforming production processes and daily life. Task allocation, the top-level design for unmanned swarm missions, is pivotal to maximizing the efficiency of the entire swarm. However, traditional optimization methods and intelligent algorithms, including reinforcement learning (RL), often struggle to adapt to the complex and unpredictable situations in these tasks. To address this challenge, we propose a novel multiagent proximal policy optimization (MAPPO) algorithm combined with the population-based learning and Gaussian mixture model (GMM)-based adjustment mechanisms (PG-MAPPO). In PG-MAPPO, the population-based learning mechanism is integrated to enable agents with diverse exploration preferences to uncover optimal collaboration patterns among unmanned aerial vehicles (UAVs), thereby enhancing cooperative efficiency. The GMM-based adjustment mechanism dynamically adjusts UAV formations for each agent, significantly improving the swarm’s flexibility and adaptability in rapidly changing environments. To demonstrate the effectiveness of PG-MAPPO, a maritime rescue simulation containing multiple complex and dynamic scenarios is conducted. Experimental results show that our algorithm achieves higher rescue success rate with faster convergence and greater stability than state-of-the-art multiagent reinforcement learning (MARL) methods in all scenarios. Notably, the PG-MAPPO algorithm improves the rescue success rate by 31.6% compared to the best-performing baseline under challenging conditions.

The widespread adoption of EVs brings practical interest to the operation optimization of the charging station. This paper considers the joint scheduling of pricing and charging control to enhance the operational capability of the station. The following contributions are made. First, a joint scheduling model of pricing and charging control is developed to maximize the expected social welfare of the charging station considering the quality of service and the price fluctuation sensitivity of EV drivers. It is formulated as a Markov decision process with a variance criterion to capture uncertainties during operation. Second, a two-phase on-line policy learning algorithm is proposed to solve this joint scheduling problem. In the first phase, it implements event-based policy iteration to find the optimal pricing scheme. In the second phase, it implements scenario-based model predictive control for smart charging under the updated pricing scheme. Third, by leveraging the performance difference theory, the optimality of the proposed algorithm is theoretically analyzed. The feasibility of the proposed method and the improved social welfare of the charging station are numerically demonstrated based on a typical charging station with distributed generation and storage. Note to Practitioners —The popularization of EVs requires the high-efficiency operation of the charging station. The joint scheduling of pricing and charging control can provide a promising way to achieve the balance between EV drivers’ satisfaction and the profit maximization of the charging station. However, it suffers from EV drivers’ uncertain responses to the pricing scheme and the coupled relationship between pricing and charging control. This multi-stage stochastic programming is non-trivial to solve. In this paper, we propose a two-phase on-line policy learning method for this joint scheduling problem. This algorithm can be implemented in the controller of the charging station. In the first phase, the event-based policy iteration can be implemented to iteratively improve the current best pricing scheme until convergence. In the second phase, the scenario-based model predictive control, which is reformulated as a mixed integer linear programming, can be quickly solved for smart charging. Case studies demonstrate the operation enhancement of the station.