IDEAS home Printed from https://ideas.repec.org/a/wsi/fracta/v30y2022i01ns0218348x2250027x.html
   My bibliography  Save this article

General Proof Of Convergence Of The Nash-Q-Learning Algorithm

Author

Listed:
  • JUN WANG

    (Command Control Engineering Institute, Army Engineering University of PLA, Nanjing 211101, P. R. China)

  • LEI CAO

    (Command Control Engineering Institute, Army Engineering University of PLA, Nanjing 211101, P. R. China)

  • XILIANG CHEN

    (Command Control Engineering Institute, Army Engineering University of PLA, Nanjing 211101, P. R. China)

  • JUN LAI

    (Command Control Engineering Institute, Army Engineering University of PLA, Nanjing 211101, P. R. China)

Abstract

In this paper, the convergence of the Nash-Q-Learning algorithm will be studied mainly. In the previous proof of convergence, each stage of the game must have a global optimal point or a saddle point. Obviously, the assumption is so strict that there are not many application scenarios for the algorithm. At the same time, the algorithm can also get a convergent result in the two Grid-World Games, which do not meet the above assumptions. Thus, previous researchers proposed that the assumptions may be appropriately relaxed. However, a rigorous theoretical proof is not given. The convergence point is a fractal attractor from the view of Fractals, general proof of convergence of the Nash-Q-Learning algorithm will be shown by the mathematical method. Meanwhile, some discussions on the efficiency and scalability of the algorithm are also described in detail.

Suggested Citation

  • Jun Wang & Lei Cao & Xiliang Chen & Jun Lai, 2022. "General Proof Of Convergence Of The Nash-Q-Learning Algorithm," FRACTALS (fractals), World Scientific Publishing Co. Pte. Ltd., vol. 30(01), pages 1-9, February.
    • Handle: RePEc:wsi:fracta:v:30:y:2022:i:01:n:s0218348x2250027x
    DOI: 10.1142/S0218348X2250027X
    as

    Download full text from publisher

    File URL: http://www.worldscientific.com/doi/abs/10.1142/S0218348X2250027X
    Download Restriction: Access to full text is restricted to subscribers
    File URL: https://libkey.io/10.1142/S0218348X2250027X?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wsi:fracta:v:30:y:2022:i:01:n:s0218348x2250027x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tai Tone Lim (email available below). General contact details of provider: https://www.worldscientific.com/worldscinet/fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.