Infinite horizon optimal control of affine nonlinear discrete switched systems using two-stage approximate dynamic programming

In this article, a novel iteration algorithm named two-stage approximate dynamic programming (TSADP) is proposed to seek the solution of nonlinear switched optimal control problem. At each iteration of TSADP, a multivariate optimal control problem is transformed to be a certain number of univariate optimal control problems. It is shown that the value function at each iteration can be characterised pointwisely by a set of smooth functions recursively obtained from TSADP, and the associated control policy, continuous control and switching control law included, is explicitly provided in a state-feedback form. Moreover, the convergence and optimality of TSADP is strictly proven. To implement this algorithm efficiently, neural networks, critic and action networks, are utilised to approximate the value function and continuous control law, respectively. Thus, the value function is expressed by the weights of critic networks pointwise. Besides, redundant weights are ruled out at each iteration to simplify the exponentially increasing computation burden. Finally, a simulation example is provided to demonstrate its effectiveness.