1 January 2020 in SIAM Journal on Control and Optimization
Optimal control problems are inherently hard to solve as the optimization must be performed simultaneously with updating the underlying system. Starting from an initial guess, Howard's policy impro...
16 November 2015 in Stochastics and Partial Differential Equations: Analysis and Computations
We prove stability and convergence of a full discretization for a class of stochastic evolution equations with super-linearly growing operators appearing in the drift term. This is done by using the recently developed tamed Euler method, which employs a fully explicit time stepping, coupled with a Galerkin scheme for the spatial discretization.
25 February 2021 in Applied Mathematics & Optimization
The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin’s optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restrictions on the time horizon of the control problem, in contrast to iterative methods based on the theory of forward-backward stochastic differential equations.
10 July 2024 in SIAM Journal on Control and Optimization
This paper studies stochastic control problems with the action space taken to be probability measures, with the objective penalized by the relative entropy. We identify a suitable metric space on which we construct a gradient flow for the measure-valued control process, in the set of admissible controls, along which the cost functional is guaranteed to decrease. It is shown that any invariant measure of this gradient flow satisfies the Pontryagin optimality principle. If the problem we work with is sufficiently convex, the gradient flow converges exponentially fast. Furthermore, the optimal measure-valued control process admits a Bayesian interpretation, which means that one can incorporate prior knowledge when solving such stochastic control problems. This work is motivated by a desire to extend the theoretical underpinning for the convergence of stochastic gradient type algorithms widely employed in the reinforcement learning community to solve control problems.
4 July 2021 in Applied Mathematical Finance
We develop several deep learning algorithms for approximating families of parametric PDE solutions. The proposed algorithms approximate solutions together with their gradients, which in the context of mathematical finance means that the derivative prices and hedging strategies are computed simultaneously. Having approximated the gradient of the solution, one can combine it with a Monte Carlo simulation to remove the bias in the deep network approximation of the PDE solution (derivative price). This is achieved by leveraging the Martingale Representation Theorem and combining the Monte Carlo simulation with the neural network. The resulting algorithm is robust with respect to the quality of the neural network approximation and consequently can be used as a black box in case only limited a-priori information about the underlying problem is available. We believe this is important as neural network-based algorithms often require fair amount of tuning to produce satisfactory results. The methods are empirically shown to work for high-dimensional problems (e.g., 100 dimensions). We provide diagnostics that shed light on appropriate network architectures.
1 January 2023 in Journal of Computational Finance
The authors propose a model called neural SDE and demonstrate how this model can make it possible to find robust bounds for the prices of derivatives and the