Model-Free Cooperative Optimal Output Regulation for Linear Discrete-Time Multi-Agent Systems Using Reinforcement Learning

In this paper, an off-policy model-free algorithm is presented for solving the cooperative optimal output regulation problem for linear discrete-time multi-agent systems. First, an adaptive distributed observer is designed for each follower to estimate the leader’s information. Then, a distributed feedback-feedforward controller is developed for each follower to solve the cooperative optimal output regulation problem utilizing the follower’s state information and the adaptive distributed observer. Based on the reinforcement learning method, an adaptive algorithm is presented to find the optimal feedback gains via online data collection from system trajectory. By designing a Sylvester map, the solution to the regulator equations is calculated via data collected from the optimal feedback gain design steps, and the feedforward control gain is found. Finally, an off-policy model-free algorithm is proposed to design the distributed feedback-feedforward controller for each follower to solve the cooperative optimal output regulation problem. A numerical example is given to verify the effectiveness of this proposed approach.