Solution Concepts for Optimal Feedback Control of Nonlinear Differential Equations: How to win against the curse of dimensionality
Karl Kunisch, RICAM and University of Graz
Optimal feedback controls for nonlinear systems are characterized by the solutions to a Hamilton Jacobi Bellmann (HJB) equation. In the deterministic case, this is a first order hyperbolic equation.
Its dimension is that of the statespace of the nonlinear system. Thus solving the HJB equation is a formidable task and one is confronted with a curse of dimensionality.
In practice, optimal feedback controls are frequently based on linearisation and subsequent treatment by efficient Riccati solvers. This can be effective, but it is local procedure, and it may fail or lead to erroneous results.
In this talk, I give a brief survey of current solution strategies to partially cope with this challenge. Subsequently I describe three approaches in some detail. The first one is a data driven technique, which approximates the solution to the HJB equation and its gradient from an ensemble of open loop solves.
The second one is based on Newton steps applied to the HJB equation. Combined with tensor calculus which allows to approximately solve HJB equations up to dimension 100. Results are shown for the control of discretized Fokker Planck equations. The third technique circumvents the direct solution of the HJB equation. Rather a neural network is trained by means of a succinctly chosen ansatz. It is proven that it approximates the optimal feedback gains as the dimension of the network is increased.
This work relies on collaborations with B.Azmi, S.Dolgov, D.Kalise, D.Vasquez, and D.Walter.