24 August 2020 in IEEE Transactions on Signal Processing
Bayesian filtering deals with computing the posterior distribution of the state of a stochastic dynamic system given noisy observations. In this paper, motivated by applications in counter-adversarial autonomous systems, we consider the following inverse filtering problem: Given a sequence of posterior distributions from a Bayesian filter, what can be inferred about the transition kernel of the state, the observation likelihoods of the sensor and the measured observations? For finite-state Markov chains observed in noise (hidden Markov models), we show that a least-squares fit for estimating the parameters and observations amounts to a combinatorial optimization problem with non-convex objective. Instead, by exploiting the algebraic structure of the corresponding Bayesian filter, we propose an algorithm based on convex optimization for reconstructing the transition kernel, the observation likelihoods and the observations. We discuss and derive conditions for identifiability. As an application of our results, we demonstrate the design of a counter-adversarial autonomous system: By observing the actions of an autonomous enemy, we estimate the accuracy of its sensors and the observations it has received. The proposed algorithms are illustrated via several numerical examples.
1 May 2016 in European Journal of Control
System identification of systems operating in closed loop is an important problem in industrial applications, where model-based control is used to an increasing extent. For model-based controllers, plant changes over time eventually result in a mismatch between the dynamics of any initial model in the controller and the actual plant dynamics. When the mismatch becomes too large, control performance suffers and it becomes necessary to re-identify the plant to restore performance. Often the available data are not informative enough when the identification is performed in closed loop and extra excitation needs to be injected. This article considers the problem of generating such excitation with the least possible disruption to the normal operations of the plant. The methods explicitly take time domain constraints into account. The formulation leads to optimal control problems which are in general very difficult optimization problems. Computationally tractable solutions based on Markov decision processes and model predictive control are presented. The performance of the suggested algorithms is illustrated in two simulation examples comparing the novel methods and algorithms available in the literature.
23 October 2018 in IEEE Transactions on Automatic Control
In identification of dynamical systems, the prediction error method (PEM) with a quadratic cost function provides asymptotically efficient estimates under Gaussian noise, but in general it requires solving a non-convex optimization problem, which may imply convergence to non-global minima. An alternative class of methods uses a non-parametric model as intermediate step to obtain the model of interest. Weighted null-space fitting (WNSF) belongs to this class, starting with the estimate of a non-parametric ARX model with least squares. Then, the reduction to a parametric model is a multi-step procedure where each step consists of the solution of a quadratic optimization problem, which can be obtained with weighted least squares. The method is suitable for both open- and closed-loop data, and can be applied to many common parametric model structures, including output-error, ARMAX, and Box-Jenkins. The price to pay is the increase of dimensionality in the non-parametric model, which needs to tend to infinity as function of the sample size for certain asymptotic statistical properties to hold. In this paper, we conduct a rigorous analysis of these properties: namely, consistency and asymptotic efficiency. Also, we perform a simulation study illustrating the performance of WNSF and identify scenarios where it can be particularly advantageous compared with state-of-the-art methods
30 August 2011 in IEEE Transactions on Automatic Control
This paper focuses on the problem of robust experiment design, i.e., how to design an input signal which gives relatively good estimation performance over a large number of systems and model structures. Specifically, we formulate the robust experiment design problem utilizing fundamental limitations on the variance of estimated parametric models as constraints. Using this formulation we design an input signal for situations where only diffuse a priori information is known about the system. Furthermore, we present a robust version of the unprejudiced optimal input design problem. To achieve this, we first develop a closed form solution for the input spectrum which minimizes the maximum weighted integral of the variance of the frequency response estimate over all model structures.
17 April 2019 in IEEE Control Systems Letters
We consider sequential stochastic decision problems in which, at each time instant, an agent optimizes its local utility by solving a stochastic program and, subsequently, announces its decision to the world. Given this action, we study the problem of estimating the agent’s private belief (i.e., its posterior distribution over the set of states of nature based on its private observations). We demonstrate that it is possible to determine the set of private beliefs that are consistent with public data by leveraging techniques from inverse optimization. We further give a number of useful characterizations of this set; for example, tight bounds by solving a set of linear programs (under concave utility). As an illustrative example, we consider estimating the private belief of an investor in regime-switching portfolio allocation. Finally, our theoretical results are illustrated and evaluated in numerical simulations.
1 April 2016 in Signal Processing