Grigory Yaremenko

Grigory Yaremenko

Auto-generated profile, based on publicly available information

Grigory Yaremenko

Skolkovo Institute of Science and Technology,Moscow,Russia

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 October 2022 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
POPavel Osinenko
Pavel Osinenko
GYGrigory Yaremenko
Grigory Yaremenko
GMGeorgiy Malaniya
Georgiy Malaniya
Control Lyapunov function is a central tool in stabilization. It generalizes an abstract energy function – a Lyapunov function – to the case of controlled systems. It is a known fact that most control Lyapunov functions are non-smooth – so is the case in non-holonomic systems, like wheeled robots and cars. Frameworks for stabilization using non-smooth control Lyapunov functions exist, like Dini aiming and steepest descent. This work generalizes the related results to the stochastic case. As the groundwork, sampled control scheme is chosen in which control actions are computed at discrete moments in time using discrete measurements of the system state. In such a setup, special attention should be paid to the sample-to-sample behavior of the control Lyapunov function. A particular challenge here is a random noise acting on the system. The central result of this work is a theorem that states, roughly, that if there is a, generally non-smooth, control Lyapunov function, the given stochastic dynamical system can be practically stabilized in the sample-and-hold mode meaning that the control actions are held constant within sampling time steps. A particular control method chosen is based on Moreau-Yosida regularization, in other words, inf-convolution of the control Lyapunov function, but the overall framework is extendable to further control schemes. It is assumed that the system noise be bounded almost surely, although the case of unbounded noise is briefly addressed.

27 February 2023 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
POPavel Osinenko
Pavel Osinenko
DDDmitrii Dobriborsci
Dmitrii Dobriborsci
GYGrigory Yaremenko
Grigory Yaremenko
GMGeorgiy Malaniya
Georgiy Malaniya
A common setting of reinforcement learning (RL) is a Markov decision process (MDP) in which the environment is a stochastic discrete-time dynamical system. Whereas MDPs are suitable in such applications as video games or puzzles, physical systems are time continuous. A general variant of RL is of digital format, where updates of the value (or cost) and policy are performed at discrete moments in time. The agent–environment loop then amounts to a sampled system, whereby sample-and-hold is a specific case. In this article, we propose and benchmark two RL methods suitable for sampled systems. Specifically, we hybridize model predictive control with critics learning the optimal Q- and value (or cost-to-go) function. Optimality is analyzed and performance comparison is done in an experimental case study with a mobile robot.

17 August 2023 in IEEE Access
Verified
Journal Article
Open access
POPavel Osinenko
Pavel Osinenko
GYGrigory Yaremenko
Grigory Yaremenko
GMGeorgiy Malaniya
Georgiy Malaniya
ABAnton Bolychev
Anton Bolychev
Online actor-critic reinforcement learning is concerned with training an agent on-the-fly via dynamic interaction with the environment. Due to the specifics of the application, it is not generally possible to perform long pre-training, as it is commonly done in off-line, tabular or Monte-Carlo mode. Such applications may be found more frequently in industry, rather than in pure digital fields, such as cloud services, video games, database management, etc., where reinforcement learning has been demonstrating success. Stability of the closed-loop of the agent plus the environment is a major challenge here, and not only in terms of the environment safety and integrity, but also in terms of sparing resources on failed training episodes. In this paper, we tackle the problem of environment stability under an actor-critic reinforcement learning agent by integration of the Lyapunov stability theory tools. Under the presented approach, the closed-loop stability is secured in all episodes without pre-training. It was observed in a case study with a mobile robot that the suggested agent could always successfully achieve the control goal, while significantly reducing the cost. While many approaches may be exploited for mobile robot control, we suggest that the experiments showed the promising potential of actor-critic reinforcement learning agents based on Lyapunov-like constraints. The presented methodology may be utilized in safety-critical, industrial applications where stability is necessary.

14 December 2021
Verified
Conference Paper
Open access
POPavel Osinenko
Pavel Osinenko
GYGrigory Yaremenko
Grigory Yaremenko
This paper addresses stochastic stabilization in case where implementation of control policies is digital, i. e., when the dynamical system is treated continuous, whereas the control actions are held constant in predefined time steps. In such a setup, special attention should be paid to the sample-to-sample behavior of the involved Lyapunov function. This paper extends on the stochastic stability results specifically to address for the sample-and-hold mode. We show that if a Markov policy stabilizes the system in a suitable sense, then it also practically stabilizes it in the sample-and-hold sense. This establishes a bridge from an idealized continuous application of the policy to its digital implementation. The central result applies to dynamical systems described by stochastic differential equations driven by the standard Brownian motion. Generalizations are discussed, including the case of non-smooth Lyapunov functions for systems driven by bounded noise. A brief overview of bounded noise models is given.

26 May 2025 in IEEE Control Systems Letters
Verified
Journal Article
Closed access
GYGrigory Yamerenko
Grigory Yamerenko
SISinan Ibrahim
Sinan Ibrahim
FMFrancisco Moreno-Mora
Francisco Moreno-Mora
POPavel Osinenko
Pavel Osinenko
SSStefan Streif
Stefan Streif
Reinforcement learning (RL) benchmarking has long relied on learning curves and cumulative reward tables, yet these metrics fail to capture critical design challenges, such as environment sensitivity, robustness, and reproducibility. This paper introduces a novel benchmarking paradigm that rigorously evaluates these understudied properties through a generative framework grounded in converse optimality. By constructing families of control problems with provably optimal solutions, our method enables statistical hypothesis testing of an algorithms response to environment perturbations both during training and post-deployment. Central to this approach is a problem generator that synthesizes parametric variations of dynamics and costs, paired with a sensitivity evaluation pipeline. To illustrate the frameworks usefulness, we apply it to compare environment sensitivity between Adam and RAdam optimizers – a case study designed to clarify how exactly the methodology provides the means to evaluate reinforcement learning algorithms in an informative way. This work contributes to RL evaluation by shifting focus from sample efficiency alone to methodologies that assess algorithms under conditions reflective of practical engineering challenges.

27 June 2025
Verified
Conference Paper
Closed access
IOIlya Osokin
Ilya Osokin
IRIlya Ryakin
Ilya Ryakin
SMSina Moghimi
Sina Moghimi
SDSergei Davidenko
Sergei Davidenko
VGVladimir Guneavoi
Vladimir Guneavoi
POPavel Osinenko
Pavel Osinenko
Yield volume estimation is the integral part of modern farming. While classical model-based approaches are already well-developed, in the recent years neural network-based end-to-end methods gain traction. This work is devoted to the case study of applying a PointNet++-like neural network to the problem of tangerine volume estimation with the point cloud data. A set of tangerines filmed by Intel RealSense camera was collected and marked. The dataset includes RGB images and depth images. The volume estimation method is run on this data presented in the form of point clouds. A number of experiments on this dataset were conducted. Experimental results suggest that the problem of volume estimation on the real data can be solved by the means of a single modern computer in real time. The output quality was assessed via comparing the prediction of the method with the real volume of the fruit. The average volume estimation error slightly exceeds 10%.