We consider a discrete-time Linear-Quadratic- Gaussian (LQG) control problem in which Massey’s directed information from the observed output of the plant to the control input is minimized while required control performance is attainable. This problem arises in several different contexts, including joint encoder and controller design for data-rate minimization in networked control systems. We show that the optimal control law is a Linear-Gaussian randomized policy. We also identify the state space realization of the optimal policy, which can be synthesized by an efficient algorithm based on semidefinite programming. Our structural result indicates that the filter-controller separation principle from the LQG control theory, and the sensor-filter separation principle from the zero-delay rate-distortion theory for Gauss-Markov sources hold simultaneously in the considered problem. A connection to the data-rate theorem for mean-square stability by Nair & Evans is also established.
17 August 2016 in IEEE Transactions on Automatic Control
Sequential rate-distortion (SRD) theory provides a framework for studying the fundamental trade-off between datarate and data-quality in real-time communication systems. In this paper, we consider the SRD problem for multi-dimensional timevarying Gauss-Markov processes under mean-square distortion criteria. We first revisit the sensor-estimator separation principle, which asserts that considered SRD problem is equivalent to a joint sensor and estimator design problem in which data-rate of the sensor output is minimized while the estimator’s performance satisfies the distortion criteria. We then show that the optimal joint design can be performed by semidefinite programming. A semidefinite representation of the corresponding SRD function is obtained. Implications of the obtained result in the context of zero-delay source coding theory and applications to networked control theory are also discussed.
8 April 2020 in IEEE Transactions on Automatic Control
We consider a dynamic traffic routing game over an urban road network involving a large number of drivers in which each driver selecting a particular route is subject to a penalty that is affine in the logarithm of the number of drivers selecting the same route. We show that the mean-field approximation of such a game leads to the so-called linearly solvable Markov decision process, implying that its mean-field equilibrium (MFE) can be found simply by solving a finite-dimensional linear system backward in time. Based on this backward-only characterization, it is further shown that the obtained MFE has the notable property of strong time-consistency. A connection between the obtained MFE and a particular class of fictitious play is also discussed.
13 September 2019 in IEEE Transactions on Automatic Control
We revisit the sequential rate-distortion (SRD) tradeoff problem for vector-valued Gauss-Markov sources with mean-squared error distortion constraints. Our study is partly motivated by the question recently raised in the paper “Rate-cost tradeoffs in control” (in Proc. 54th Annu. Allerton Conf. Commun., Control, Comput., 2016, pp. 1157-1164) regarding the correctness of its solution algorithm known in the literature. We show via a counterexample that the dynamic reverse water-filling algorithm suggested by (15) of the paper “Stochastic linear control over a communication channel” (IEEE Trans. Autom. Control, vol. 49, pp. 1549-1561, 2004) is not applicable to this problem, and consequently, the closed-form expression of the asymptotic SRD function derived in (17) of the paper “Stochastic linear control over a communication channel” (IEEE Trans. Autom. Control, vol. 49, pp. 1549-1561, 2004) is not correct in general. Nevertheless, we show that the multidimensional Gaussian SRD function is semidefinite representable, and thus, it is readily computable.
8 September 2021 in IEEE Transactions on Automatic Control
In this article, we revisit the sequential source-coding framework to analyze fundamental performance limitations of discrete-time stochastic control systems subject to feedback data-rate constraints in finite-time horizon. The basis of our results is a new characterization of the lower bound on the minimum total-rate achieved by sequential codes subject to a total (across time) distortion constraint and a computational algorithm that allocates optimally the rate-distortion, for a given distortion level, at each instant of time and any fixed finite-time horizon. The idea behind this characterization facilitates the derivation of analytical , nonasymptotic , and finite-dimensional lower and upper bounds in two control-related scenarios: a) A parallel time-varying Gauss–Markov process with identically distributed spatial components that are quantized and transmitted through a noiseless channel to a minimum mean-squared error decoder; and b) a time-varying quantized linear quadratic Gaussian (LQG) closed-loop control system, with identically distributed spatial components and with a random data-rate allocation. Our nonasymptotic lower bound on the quantized LQG control problem reveals the absolute minimum data-rates for (mean square) stability of our time-varying plant for any fixed finite-time horizon. We supplement our framework with illustrative simulation experiments.
16 June 2022 in IEEE Transactions on Automatic Control
We study the problem of synthesizing a controller that maximizes the entropy of a partially observable Markov decision process (POMDP) subject to a constraint on the expected total reward. Such a controller minimizes the predictability of an agent’s trajectories to an outside observer while guaranteeing the completion of a task expressed by a reward function. Focusing on finite-state controllers (FSCs) with deterministic memory transitions, we show that the maximum entropy of a POMDP is lower bounded by the maximum entropy of the parametric Markov chain (pMC) induced by such FSCs. This relationship allows us to recast the entropy maximization problem as a so-called parameter synthesis problem for the induced pMC. We then present an algorithm to synthesize an FSC that locally maximizes the entropy of a POMDP over FSCs with the same number of memory states. In a numerical example, we highlight the benefit of using an entropy-maximizing FSC compared to an FSC that simply finds a feasible policy for accomplishing a task.