The problem of the mean square exponential stability for a class of discrete-time linear stochastic systems subject to independent random perturbations and Markovian switching is investigated. Four different definitions of the concept of exponential stability in mean square are introduced and it is shown that they are not always equivalent. One definition of the concept of mean square exponential stability is done in terms of the exponential stability of the evolution defined by a sequence of linear positive operators on an ordered Hilbert space. The other three definitions are given in terms of different types of exponential behavior of the trajectories of the considered system. In our approach the Markov chain is not prefixed. The only available information about the Markov chain is the sequence of probability transition matrices and the set of its states. In this way one obtains that if the system is affected by Markovian jumping the property of exponential stability is independent of the initial distribution of the Markov chain. The definition expressed in terms of exponential stability of the evolution generated by a sequence of linear positive operators, allows us to characterize the mean square exponential stability based on the existence of some quadratic Lyapunov functions. Unlike the continuous time framework, for the discrete time linear stochastic systems with Markovian jumping two types of Lyapunov operators are introduced. Therefore in the case of discrete time linear stochastic systems subject to Markovian perturbations one obtains characterizations of the mean square exponential stability which do not have an analogous in the continuous time. One of the aim of this paper is to show that in the general case of discretetime time-varying linear stochastic systems subject to an homogeneous or an inhomogeneous Markov chain, exponential stability mean square defined in terms of state space trajectories of the systems cannot be always characterized via quadratic Lyapunov functions. The results developed in this paper may be used to derive some procedures for designing stabilizing controllers for the considered class of discrete-time linear stochastic systems.

In this paper, the problem of stochastic observability and stochastic detectability of a class of discrete-time linear systems affected by independent random perturbations and Markovian jumping is investigated. The definition of stochastic observability adopted here extends to this framework the definition of the wellknown uniform observability of a deterministic discrete-time time-varying linear system. By several examples, we show that the concept of stochastic observability introduced in this paper is less restrictive than those introduced in other existing works and it does not imply always the stochastic detectability as it would be expected. Finally, one proves that this kind of stochastic observability allows us to derive a Barbashin–Krasovskii-type result for exponential stability in mean square.

This letter investigates an infinite-horizon incentive Stackelberg game for discrete-time linear stochastic systems subject to Markov jump parameters and external disturbance by means of static output-feedback (SOF). In contrast to the existing studies, players can only have access to local output information in designing their incentive Stackelberg strategies with H _∞ constraint. It is shown that the incentive Stackelberg strategy set is obtained by solving a set of higher-order cross-coupled Lyapunov type equations. As another important contribution, a numerical algorithm is proposed based on the coordinate descent method that guarantees local convergence. A numerical example demonstrates the existence of the SOF incentive Stackelberg strategy set and the effectiveness of the proposed algorithm.

This article investigates the problem of the definition and computation of an H₂-type norm for discrete-time time-varying periodic stochastic linear systems simultaneously affected by multiplicative white noise perturbations and random jumping according to a Markov chain with an infinite countable number of states. Also, we solve an optimization problem that contains, as a special case, the H₂ optimal control problem for the considered class of stochastic systems under the assumption of perfect state measurements.