IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i16p2555-d1458849.html
   My bibliography  Save this article

Optimal Asymptotic Tracking Control for Nonzero-Sum Differential Game Systems with Unknown Drift Dynamics via Integral Reinforcement Learning

Author

Listed:
  • Chonglin Jing

    (Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China)

  • Chaoli Wang

    (Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China)

  • Hongkai Song

    (Equipment Assets Management Office, Shanghai Jian Qiao University, Shanghai 201306, China)

  • Yibo Shi

    (Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China)

  • Longyan Hao

    (Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China)

Abstract

This paper employs an integral reinforcement learning (IRL) method to investigate the optimal tracking control problem (OTCP) for nonlinear nonzero-sum (NZS) differential game systems with unknown drift dynamics. Unlike existing methods, which can only bound the tracking error, the proposed approach ensures that the tracking error asymptotically converges to zero. This study begins by constructing an augmented system using the tracking error and reference signal, transforming the original OTCP into solving the coupled Hamilton–Jacobi (HJ) equation of the augmented system. Because the HJ equation contains unknown drift dynamics and cannot be directly solved, the IRL method is utilized to convert the HJ equation into an equivalent equation without unknown drift dynamics. To solve this equation, a critic neural network (NN) is employed to approximate the complex value function based on the tracking error and reference information data. For the unknown NN weights, the least squares (LS) method is used to design an estimation law, and the convergence of the weight estimation error is subsequently proven. The approximate solution of optimal control converges to the Nash equilibrium, and the tracking error asymptotically converges to zero in the closed system. Finally, we validate the effectiveness of the proposed method in this paper based on MATLAB using the ode45 method and least squares method to execute Algorithm 2.

Suggested Citation

  • Chonglin Jing & Chaoli Wang & Hongkai Song & Yibo Shi & Longyan Hao, 2024. "Optimal Asymptotic Tracking Control for Nonzero-Sum Differential Game Systems with Unknown Drift Dynamics via Integral Reinforcement Learning," Mathematics, MDPI, vol. 12(16), pages 1-21, August.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2555-:d:1458849
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/16/2555/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/16/2555/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Clemhout, Simone & Wan, Henry Jr., 1994. "Differential games -- Economic applications," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 2, chapter 23, pages 801-825, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gerhard Sorger, 2005. "A dynamic common property resource problem with amenity value and extraction costs," International Journal of Economic Theory, The International Society for Economic Theory, vol. 1(1), pages 3-19, March.
    2. Julio Huato, 2023. "Inequality and Growth: A Two-Player Dynamic Game with Production and Appropriation," Papers 2304.01855, arXiv.org.
    3. Bethmann, Dirk, 2008. "The open-loop solution of the Uzawa-Lucas model of endogenous growth with N agents," Journal of Macroeconomics, Elsevier, vol. 30(1), pages 396-414, March.
    4. Leong, Chee Kian, 2008. "Capitalism and Economic Growth: A Game-Theoretic Perspective," MPRA Paper 10472, University Library of Munich, Germany.
    5. Engelbert Dockner & Florian Wagener, 2014. "Markov perfect Nash equilibria in models with a single capital stock," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 56(3), pages 585-625, August.
    6. Hassan Benchekroun & Ngo Van Long, 2001. "Leader and Follower: A Differential Game Model," CIRANO Working Papers 2001s-08, CIRANO.
    7. Zaruhi Hakobyan & Christos Koulovatianos, 2021. "Symmetric Markovian Games of Commons with Potentially Sustainable Endogenous Growth," Dynamic Games and Applications, Springer, vol. 11(1), pages 54-83, March.
    8. Markus K. Brunnermeier & Lasse Heje Pedersen, 2005. "Predatory Trading," Journal of Finance, American Finance Association, vol. 60(4), pages 1825-1863, August.
    9. Piga, Claudio A. G., 2000. "Competition in a duopoly with sticky price and advertising," International Journal of Industrial Organization, Elsevier, vol. 18(4), pages 595-614, May.
    10. Richard Cornes & Ngo Van Long & Koji Shimomura, 2000. "Strategic Behavior under Intertemporal Production Externalities," CIRANO Working Papers 2000s-07, CIRANO.
    11. Giorgio Fabbri & Silvia Faggian & Giuseppe Freni, 2022. "On competition for spatially distributed resources in networks: an extended version," Working Papers 2022:03, Department of Economics, University of Venice "Ca' Foscari".
    12. repec:bla:econom:v:69:y:2002:i:274:p:207-21 is not listed on IDEAS
    13. Ngo Van Long & Koji Shimomura & Harutaka Takahashi, 1999. "Comparing Open-loop With Markov Equilibria in a Class of Differential Games," The Japanese Economic Review, Japanese Economic Association, vol. 50(4), pages 457-469, December.
    14. Cornes, Richard & Van Long, Ngo & Shimomura, Koji, 2001. "Drugs and pests: intertemporal production externalities," Japan and the World Economy, Elsevier, vol. 13(3), pages 255-278, August.
    15. Salo, Seppo & Tahvonen, Olli, 2001. "Oligopoly equilibria in nonrenewable resource markets," Journal of Economic Dynamics and Control, Elsevier, vol. 25(5), pages 671-702, May.
    16. Murray C. Kemp & Ngo Van Long, 2007. "Development Aid in the Presence of Corruption: Differential Games among Donors," CIRANO Working Papers 2007s-23, CIRANO.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2555-:d:1458849. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.