This paper mainly proposes some improved stochastic gradient descent (SGD) algorithms with a fractional order gradient for the online optimization problem. For three scenarios, including standard learning rate, adaptive gradient learning rate, and momentum learning rate, three new SGD algorithms are designed combining a fractional order gradient and it is shown that the corresponding regret functions are convergent at a sub-linear rate. Then we discuss the impact of the fractional order on the convergence and monotonicity and prove that the better performance can be obtained by adjusting the order of the fractional gradient. Finally, several practical examples are given to verify the superiority and validity of the proposed algorithm.
8 February 2024 in International Journal of Control, Automation, and Systems
In this paper, we address the finite-time stability problem of Caputo nabla fractional-order switched linear systems with . Firstly, the monotonicity of the discrete Mittag-Leffler function is proposed. Secondly, under the per-designed switching rules, the form of the solution for Caputo nabla fractional-order switched linear systems is obtained by using the discrete unit step function. On the above basis, some sufficient conditions of finite-time stability for Caputo nabla fractional-order switched linear systems are proposed, according to the discrete Grönwall inequality and the monotonicity of the discrete Mittag-Leffler function. Finally, simulation verification is carried out via three numerical examples.
This paper investigates 1-moment exponential stability and exponential mean-square stability (EMS stability) under average dwell time (ADT) and the preset deterministic switching mechanism of dual switching linear continuous-time positive systems when a numerical realization does not exist. The signs of subsystem matrices, but not their structures of magnitude, are key information that causes a qualitative concept of stability called sign stability. Both 1-moment exponential stability and EMS stability, which are the traditional stability concepts, are generalized intrinsically. Hence, both 1-moment exponential sign stability and EMS sign stability are introduced and are proven based on sign equivalency. It is shown that they are symmetrically and qualitatively stable. Notably, the notion of stability can be checked quantitatively using some examples.