Optimal Adaptive Control Methods for Structurally Varying Systems

The problem of simultaneously identifying and controlling a time-varying, perfectly-observed linear system is posed. The parameters are assumed to obey a Markov structure and are estimated with a Kalman filter. The problem can be solved conceptually by dynamic programming, but even with a quadratic loss function the analytical computations cannot be carried out for more than one step because of the dual nature of the optimal control law. All approximations to the solution that have been proposed in the literature, and two approximations that are presented here for the first time are analyzed. They are classified into dual and non-dual methods. Analytical comparison is untractable; hence Monte Carlo simulations are used. A set of experiments is presented in which five non-dual methods are compared. The numerical results indicate a possible ordering among these approximations.