IDEAS home Printed from https://ideas.repec.org/a/spr/dyngam/v12y2022i2d10.1007_s13235-021-00399-8.html
   My bibliography  Save this article

Newton’s Method, Bellman Recursion and Differential Dynamic Programming for Unconstrained Nonlinear Dynamic Games

Author

Listed:
  • Bolei Di
  • Andrew Lamperski

Abstract

Dynamic games arise when multiple agents with differing objectives control a dynamic system. They model a wide variety of applications in economics, defense, energy systems and etc. However, compared to single-agent control problems, the computational methods for dynamic games are relatively limited. As in the single-agent case, only specific dynamic games can be solved exactly, so approximation algorithms are required. In this paper, we show how to extend the Newton step algorithm, the Bellman recursion and the popular differential dynamic programming (DDP) for single-agent optimal control to the case of full-information nonzero sum dynamic games. We show that the Newton’s step can be solved in a computationally efficient manner and inherits its original quadratic convergence rate to open-loop Nash equilibria, and that the approximated Bellman recursion and DDP methods are very similar and can be used to find local feedback $$O(\varepsilon ^2)$$ O ( ε 2 ) -Nash equilibria. Numerical examples are provided.

Suggested Citation

  • Bolei Di & Andrew Lamperski, 2022. "Newton’s Method, Bellman Recursion and Differential Dynamic Programming for Unconstrained Nonlinear Dynamic Games," Dynamic Games and Applications, Springer, vol. 12(2), pages 394-442, June.
    • Handle: RePEc:spr:dyngam:v:12:y:2022:i:2:d:10.1007_s13235-021-00399-8
    DOI: 10.1007/s13235-021-00399-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s13235-021-00399-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s13235-021-00399-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Suresh P. Sethi, 2021. "Optimal Control Theory," Springer Texts in Business and Economics, Springer, edition 4, number 978-3-030-91745-6, December.
    2. Steffan Berridge & Jacek Krawczyk, "undated". "Relaxation Algorithms in Finding Nash Equilibrium," Computing in Economics and Finance 1997 159, Society for Computational Economics.
    3. Alberto Bressan & Khai T. Nguyen, 2018. "Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games," Dynamic Games and Applications, Springer, vol. 8(1), pages 42-78, March.
    4. Alain Haurie & Jacek B Krawczyk & Georges Zaccour, 2012. "Games and Dynamic Games," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 8442, February.
    5. Krawczyk, Jacek B., 2005. "Coupled constraint Nash equilibria in environmental games," Resource and Energy Economics, Elsevier, vol. 27(2), pages 157-181, June.
    6. Jacob Engwerda, 2017. "A Numerical Algorithm to Calculate the Unique Feedback Nash Equilibrium in a Large Scalar LQ Differential Game," Dynamic Games and Applications, Springer, vol. 7(4), pages 635-656, December.
    7. R. Buckdahn & P. Cardaliaguet & M. Quincampoix, 2011. "Some Recent Aspects of Differential Game Theory," Dynamic Games and Applications, Springer, vol. 1(1), pages 74-114, March.
    8. Engwerda, J.C., 2000. "Feedback Nash equilibria in the scalar infinite horizon LQ-Game," Other publications TiSEM 58ccf964-4ca1-4d67-9a68-a, Tilburg University, School of Economics and Management.
    9. Jacek Krawczyk, 2007. "Numerical solutions to coupled-constraint (or generalised Nash) equilibrium problems," Computational Management Science, Springer, vol. 4(2), pages 183-204, April.
    10. repec:dau:papers:123456789/6046 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elnaz Kanani Kuchesfehani & Georges Zaccour, 2015. "S-adapted Equilibria in Games Played Over Event Trees with Coupled Constraints," Journal of Optimization Theory and Applications, Springer, vol. 166(2), pages 644-658, August.
    2. J. Contreras & J. B. Krawczyk & J. Zuccollo, 2016. "Economics of collective monitoring: a study of environmentally constrained electricity generators," Computational Management Science, Springer, vol. 13(3), pages 349-369, July.
    3. Contreras, Javier & Krawczyk, Jacek & Zuccollo, James, 2008. "Can planners control competitive generators?," MPRA Paper 10395, University Library of Munich, Germany.
    4. Francisco Facchinei & Christian Kanzow, 2010. "Generalized Nash Equilibrium Problems," Annals of Operations Research, Springer, vol. 175(1), pages 177-211, March.
    5. Jacek B. Krawczyk & Mabel Tidball, 2016. "Economic Problems with Constraints: How Efficiency Relates to Equilibrium," International Game Theory Review (IGTR), World Scientific Publishing Co. Pte. Ltd., vol. 18(04), pages 1-19, December.
    6. Boucekkine, Raouf & Krawczyk, Jacek B. & Vallée, Thomas, 2010. "Towards an understanding of tradeoffs between regional wealth, tightness of a common environmental constraint and the sharing rules," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1813-1835, September.
    7. Flam, Sjur & Ruszczynski, A., 2006. "Computing Normalized Equilibria in Convex-Concave Games," Working Papers 2006:9, Lund University, Department of Economics.
    8. Krawczyk, Jacek & Zuccollo, James, 2006. "NIRA-3: An improved MATLAB package for finding Nash equilibria in infinite games," MPRA Paper 1119, University Library of Munich, Germany.
    9. Contreras, Javier & Krawczyk, Jacek & Zuccollo, James, 2008. "The invisible polluter: Can regulators save consumer surplus?," MPRA Paper 9890, University Library of Munich, Germany.
    10. Krawczyk, Jacek & Azzato, Jeffrey, 2006. "NISOCSol an algorithm for approximating Markovian equilibria in dynamic games with coupled-constraints," MPRA Paper 1195, University Library of Munich, Germany.
    11. Yann BRAOUEZEC & Keyvan KIANI, 2021. "Economic foundations of generalized games with shared constraint: Do binding agreements lead to less Nash equilibria?," Working Papers 2021-ACF-06, IESEG School of Management.
    12. Han, Deren & Zhang, Hongchao & Qian, Gang & Xu, Lingling, 2012. "An improved two-step method for solving generalized Nash equilibrium problems," European Journal of Operational Research, Elsevier, vol. 216(3), pages 613-623.
    13. Masao Fukushima, 2011. "Restricted generalized Nash equilibria and controlled penalty algorithm," Computational Management Science, Springer, vol. 8(3), pages 201-218, August.
    14. Dane A. Schiro & Benjamin F. Hobbs & Jong-Shi Pang, 2016. "Perfectly competitive capacity expansion games with risk-averse participants," Computational Optimization and Applications, Springer, vol. 65(2), pages 511-539, November.
    15. A. Heusinger & C. Kanzow, 2009. "Relaxation Methods for Generalized Nash Equilibrium Problems with Inexact Line Search," Journal of Optimization Theory and Applications, Springer, vol. 143(1), pages 159-183, October.
    16. K. Kubota & M. Fukushima, 2010. "Gap Function Approach to the Generalized Nash Equilibrium Problem," Journal of Optimization Theory and Applications, Springer, vol. 144(3), pages 511-531, March.
    17. Koichi Nabetani & Paul Tseng & Masao Fukushima, 2011. "Parametrized variational inequality approaches to generalized Nash equilibrium problems with shared constraints," Computational Optimization and Applications, Springer, vol. 48(3), pages 423-452, April.
    18. Gürkan, G. & Pang, J.S., 2009. "Approximizations of Nash equilibria," Other publications TiSEM de211d31-d77d-4211-9ca8-2, Tilburg University, School of Economics and Management.
    19. Krawczyk, Jacek B & Townsend, Wilbur, 2014. "NIRA-GUI: A matlab application which solves for couple-constraint nash equibria from a symbolic specification," Working Paper Series 3414, Victoria University of Wellington, School of Economics and Finance.
    20. Jacob Engwerda, 2022. "Min-Max Robust Control in LQ-Differential Games," Dynamic Games and Applications, Springer, vol. 12(4), pages 1221-1279, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:dyngam:v:12:y:2022:i:2:d:10.1007_s13235-021-00399-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.