27 March 2015 in IEEE Transactions on Automatic Control
We consider the problem of risk-sensitive stochastic control under a Markov modulated denial-of-service (DoS) attack strategy in which the attacker, using a hidden Markov model, stochastically jams the control packets in the system. For a discrete-time partially observed stochastic system with an exponential running cost, we provide a solution in terms of the finite-dimensional dynamics of the system through a chain of measure transformation techniques. We also prove a separation principle under which a recursive optimal control policy together with a newly defined information-state constitutes an equivalent completely observable stochastic control problem. Remarkably, on the transformed measure space, the solution to the optimal control problem appears as if it depends only on the sample-path (or path-estimation) of the DoS attack sequences in the system.
1 October 2017 in International Journal of Heat and Mass Transfer
1 January 2015 in SIAM Journal on Control and Optimization
In this paper, we consider a diffusion process pertaining to a distributed control system formed by a chain of $n$ subsystems (with $n \ge 2$), where a random perturbation enters only in the first subsystem and is then subsequently transmitted to the other subsystems. We also assume that, for any $\ell \in \{2, \ldots, n\}$, the distributed control system formed by the first $\ell$ subsystems (i.e., from the first subsystem up to the $\ell$th subsystem) satisfies an appropriate Hörmander condition. As a result, the diffusion process is degenerate, in the sense that the backward operator associated with it is a degenerate parabolic equation. In particular, we consider the following two problems: (i) We provide an asymptotic estimate for the exit probability with which the diffusion process (corresponding to a particular subsystem) exits from a given bounded open domain during a certain time interval; the approach for such an asymptotic estimate basically relies on the interpretation of the exit probability function as a value function for a family of stochastic control problems that are associated with the underlying chain of distributed control system with small random perturbations. (ii) We establish a connection between the minimum exit rate with which the diffusion process exits from the given bounded open domain and the principal eigenvalue for the infinitesimal generator with zero boundary conditions. Such a connection also allows us to derive a family of Hamilton--Jacobi--Bellman equations for which we provide a verification theorem that shows the validity of the corresponding optimal control problems. Moreover, we provide an estimate for the attainable exit probability of the diffusion process with respect to a set of admissible optimal Markov controls for the optimal control problems.
14 September 2009 in International Journal of Control