Dmitrii Dobriborsci

Dmitrii Dobriborsci

Auto-generated profile, based on publicly available information

Dmitrii Dobriborsci

Deggendorf Institute of Technology

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 September 2021 in IEEE Access
Verified
Journal Article
Open access
POPavel Osinenko
Pavel Osinenko
DDDmitrii Dobriborsci
Dmitrii Dobriborsci
Plain reinforcement learning (RL) may be prone to loss of convergence, constraint violation, unexpected performance, etc. Commonly, RL agents undergo extensive learning stages to achieve proper functionality. This is in contrast to classical control algorithms, which are typically model-based. A direction of research is the fusion of RL with such algorithms, especially model-predictive control (MPC). This, however, introduces new hyper-parameters related to the prediction horizon. Furthermore, RL is usually concerned with Markov decision processes. Nevertheless, most of the real environments are not time-discrete. The factual physical setting of RL consists of a digital agent and a time-continuous dynamical system. There is thus, in fact, yet another hyper-parameter – the agent sampling time. In this paper, we investigate the effects of prediction horizon and sampling of two hybrid RL-MPC agents in a case study with a mobile robot parking, which is, in turn, a canonical control problem. We benchmark the agents with a simple variant of MPC. The sampling showed a “sweet spot” behavior, whereas the RL agents demonstrated merits at shorter horizons.

31 August 2023 in Journal of Intelligent Manufacturing
Verified
Journal Article
Closed access
DDDmitrii Dobriborsci
Dmitrii Dobriborsci
RZRoman Zashchitin
Roman Zashchitin
MKMikhail Kakanov
Mikhail Kakanov
WAWolfgang Aumer
Wolfgang Aumer
POPavel Osinenko
Pavel Osinenko

27 February 2023 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
POPavel Osinenko
Pavel Osinenko
DDDmitrii Dobriborsci
Dmitrii Dobriborsci
GYGrigory Yaremenko
Grigory Yaremenko
GMGeorgiy Malaniya
Georgiy Malaniya
A common setting of reinforcement learning (RL) is a Markov decision process (MDP) in which the environment is a stochastic discrete-time dynamical system. Whereas MDPs are suitable in such applications as video games or puzzles, physical systems are time continuous. A general variant of RL is of digital format, where updates of the value (or cost) and policy are performed at discrete moments in time. The agent–environment loop then amounts to a sampled system, whereby sample-and-hold is a specific case. In this article, we propose and benchmark two RL methods suitable for sampled systems. Specifically, we hybridize model predictive control with critics learning the optimal Q- and value (or cost-to-go) function. Optimality is analyzed and performance comparison is done in an experimental case study with a mobile robot.

1 June 2018
Verified
Conference Paper
Closed access
SVSergey M. Vlasov
Sergey M. Vlasov
AKAleksandra S. Kirsanova
Aleksandra S. Kirsanova
DDDmitrii Dobriborsci
Dmitrii Dobriborsci
OBOleg I. Borisov
Oleg I. Borisov
VGVladislav S. Gromov
Vladislav S. Gromov
ASArtem N. Semenev
Artem N. Semenev
The problem of controlling for robotic boat with unknown parameters and unmeasurable velocity and acceleration is addressed in this paper. The controller design is based on the output robust control law. It was modified with adaptation algorithms, which allowed to eliminate a static error despite parametric and functional uncertainties. As result, the regulator minimized static error compared to robust controller. The efficiency of the proposed algorithm was illustrated by the experimental approval using the robotic boat model in the task of ellipse path movement. There is comparison between of robust controller and adaptive version in the paper.

14 April 2017 in Scientific and Technical Journal of Information Technologies, Mechanics and Optics
Verified
Journal Article
Open access
ABA.A. Bobtsov
A.A. Bobtsov
DDD. Dobriborsci
D. Dobriborsci
AKA.A. Kapitonov
A.A. Kapitonov
NAVIGATION AND CONTROL SYSTEM FOR MOBILE ROBOT Journal Scientific and Technical Of Information Technologies, Mechanics and Optics

23 August 2021 in arXiv
Verified
Preprint
Open access
PDF file
DDDmitrii Dobriborsci
Dmitrii Dobriborsci
POPavel Osinenko
Pavel Osinenko
Reinforcement learning (RL) has been successfully used in various simulations and computer games. Industry-related applications, such as autonomous mobile robot motion control, are somewhat challenging for RL up to date though. This paper presents an experimental evaluation of predictive RL controllers for optimal mobile robot motion control. As a baseline for comparison, model-predictive control (MPC) is used. Two RL methods are tested: a roll-out Q-learning, which may be considered as MPC with terminal cost being a Q-function approximation, and a so-called stacked Q-learning, which in turn is like MPC with the running cost substituted for a Q-function approximation. The experimental foundation is a mobile robot with a differential drive (Robotis Turtlebot3). Experimental results showed that both RL methods beat the baseline in terms of the accumulated cost, whereas the stacked variant performed best. Provided the series of previous works on stacked Q-learning, this particular study supports the idea that MPC with a running cost adaptation inspired by Q-learning possesses potential of performance boost while retaining the nice properties of MPC.