We develop a method for determining whether a stochastic system is safe, i.e., whether its trajectories reach unsafe states. Specifically, we define and solve a probabilistic safety problem for Markov processes. Based on the knowledge of the extended generator, we are able to develop an evolution equation, as a system of integral equations, describing the connection between unsafe and initial states. Subsequently, using the moment method, we approximate the infinite-dimensional optimisation problem searching for the largest set of safe states by a finite dimensional polynomial optimisation problem. In particular, we address the above safety problem to a special class of stochastic hybrid processes, namely piecewise-deterministic Markov processes. These are characterized by deterministic dynamics and stochastic jumps, where both the time and the destination of the jumps are stochastic. In addition, the jumps can be both spontaneous (in the style of a Poisson process) or forced (governed by guards). In this case, the extended generator of this process and its corresponding martingale problem turn out to be defined on a rather restricted domain. To circumvent this
This work establishes an optimisation scheme for computing safety of stochastic hybrid systems. Specifically, the scheme computes the probability of reaching the set of unsafe states from a set of initial states. We address hybrid systems with both sporadic and state dependent switches. The method combines two approaches: the barrier certificate method and potential theory.
In this letter, we study the so-called $\mathbf {p}$ -safety of a Markov chain. We say that a state is $\mathbf {p}$ -safe in a state space $S$ with respect to an unsafe set $U$ if the process stays in the state space and hits the set $U$ with the probability less than $\mathbf {p}$ . We show several ways of computing $\mathbf {p}$ -safety: by means the Dirichlet problem, the evolution equation, the barrier certificates, and the Martin kernel. The set of barrier certificates forms a cone. We show how to generate barrier certificates from the set of extreme points of a cone base.
4 July 2023 in IEEE Transactions on Automatic Control
This paper aims to incorporate safety specifications into Markov decision processes. Explicitly, we address the minimization problem up to a stopping time with safety constraints. We establish a formalism leaning upon the evolution equation to achieve our goal. We show how to compute the safety function with dynamic programming. In the last part of the paper, we develop several algorithms for safe stochastic optimisation using linear and dynamic programming.