This paper addresses the problem of defining a day-ahead consumption plan for charging a fleet of electric vehicles (EVs), and following this plan during operation. A challenge herein is the beforehand unknown charging flexibility of EVs, which depends on numerous details about each EV (e.g., plug-in times, power limitations, battery size, power curve, etc.). To cope with this challenge, EV charging is controlled during opertion by a heuristic scheme, and the resulting charging behavior of the EV fleet is learned by using batch mode reinforcement learning. Based on this learned behavior, a cost-effective day-ahead consumption plan can be defined. In simulation experiments, our approach is benchmarked against a multistage stochastic programming solution, which uses an exact model of each EVs charging flexibility. Results show that our approach is able to find a day-ahead consumption plan with comparable quality to the benchmark solution, without requiring an exact day-ahead model of each EVs charging flexibility.
29 November 2012 in IEEE Transactions on Smart Grid
In this paper, we present a scalable approach for DSM (demand side management) of PHEVs (plug-in hybrid electric vehicles). Essentially, our approach consists of three steps: aggregation, optimization, and control. In the aggregation step, individual PHEV charging constraints are aggregated upwards in a tree structure. In the optimization step, the aggregated constraints are used for scalable computation of a collective charging plan, which minimizes costs for electricity supply. In the real-time control step, this charging plan is used to create an incentive signal for all PHEVs, determined by a market-based priority scheme. These three steps are executed iteratively to cope with uncertainty and dynamism. In simulation experiments, the proposed three-step approach is benchmarked against classic, fully centralized approaches. Results show that our approach is able to charge PHEVs with comparable quality to optimal, centrally computed charging plans, while significantly improving scalability.
Motivated by recent developments in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL in energy management in microgrids. We tackle the challenge of finding a closed-loop control policy to optimally schedule the operation of a storage device, in order to maximize self-consumption of local photovoltaic production in a microgrid. In this work, the fitted Q-iteration algorithm, a standard batch RL technique, is used by an RL agent to construct a control policy. The proposed method is data-driven and uses a state-action value function to find an optimal scheduling plan for a battery. The battery’s charge and discharge efficiencies, and the nonlinearity in the microgrid due to the inverter’s efficiency are taken into account. The proposed approach has been tested by simulation in a residential setting using data from Belgian residential consumers. The developed framework is benchmarked with a model-based technique, and the simulation results show a performance gap of 19%. The simulation results provide insight for developing optimal policies in more realistically-scaled and interconnected microgrids and for including uncertainties in generation and consumption for which white-box models become inaccurate and/or infeasible.
An integrating sphere-based setup to obtain a quick and reliable determination of the internal quantum efficiency of strongly scattering luminescent materials is presented. In literature, two distinct but similar measurement procedures are frequently mentioned: a “two measurement” and a “three measurement” approach. Both methods are evaluated by applying the rigorous integrating sphere theory. It was found that both measurement procedures are valid. Additionally, the two methods are compared with respect to the uncertainty budget of the obtained values of the quantum efficiency. An inter-laboratory validation using the two distinct procedures was performed. The conclusions from the theoretical study were confirmed by the experimental data.