Harri Aaltonen

Mr. Harri Aaltonen

Department of Electrical Engineering and Automation, School of Electrical Engineering, Aalto Universi-ty
orcid

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 July 2022 in Energies
Verified
Journal Article
Open access
PDF file
HAHarri Aaltonen
Harri Aaltonen
Seppo SierlaSeppo Sierla
Seppo Sierla
VKVille Kyrki
Ville Kyrki
Mahdi Pourakbari-KasmaeiMahdi Pourakbari-Kasmaei
Mahdi Pourakbari-Kasmaei
VVValeriy Vyatkin
Valeriy Vyatkin
Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multi-objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physics-based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.

6 September 2021 in Energies
Verified
Journal Article
Open access
PDF file
HAHarri Aaltonen
Harri Aaltonen
Seppo SierlaSeppo Sierla
Seppo Sierla
Rakshith SubramanyaRakshith Subramanya
Rakshith Subramanya
VVValeriy Vyatkin
Valeriy Vyatkin
Battery storages are an essential element of the emerging smart grid. Compared to other distributed intelligent energy resources, batteries have the advantage of being able to rapidly react to events such as renewable generation fluctuations or grid disturbances. There is a lack of research on ways to profitably exploit this ability. Any solution needs to consider rapid electrical phenomena as well as the much slower dynamics of relevant electricity markets. Reinforcement learning is a branch of artificial intelligence that has shown promise in optimizing complex problems involving uncertainty. This article applies reinforcement learning to the problem of trading batteries. The problem involves two timescales, both of which are important for profitability. Firstly, trading the battery capacity must occur on the timescale of the chosen electricity markets. Secondly, the real-time operation of the battery must ensure that no financial penalties are incurred from failing to meet the technical specification. The trading-related decisions must be done under uncertainties, such as unknown future market prices and unpredictable power grid disturbances. In this article, a simulation model of a battery system is proposed as the environment to train a reinforcement learning agent to make such decisions. The system is demonstrated with an application of the battery to Finnish primary frequency reserve markets.