IDEAS home Printed from https://ideas.repec.org/a/spr/dyngam/v2y2012i3p294-312.html
   My bibliography  Save this article

Turnpike Theorems for Markov Games

Author

Listed:
  • Vassili Kolokoltsov
  • Wei Yang

Abstract

This paper has a two-fold purpose. First, we attempt to outline the development of the turnpike theorems in the last several decades. Second, we study turnpike theorems in finite-horizon two-person zero-sum Markov games on a general Borel state space. Utilising the Bellman (or Shapley) operator defined for this game, we prove stochastic versions of the early turnpike theorem on the set of optimal strategies and the middle turnpike theorem on the distribution of the state space. Copyright Springer Science+Business Media, LLC 2012

Suggested Citation

  • Vassili Kolokoltsov & Wei Yang, 2012. "Turnpike Theorems for Markov Games," Dynamic Games and Applications, Springer, vol. 2(3), pages 294-312, September.
    • Handle: RePEc:spr:dyngam:v:2:y:2012:i:3:p:294-312
    DOI: 10.1007/s13235-012-0047-6
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s13235-012-0047-6
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s13235-012-0047-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nowak, Andrzej S., 2008. "Equilibrium in a dynamic game of capital accumulation with the overtaking criterion," Economics Letters, Elsevier, vol. 99(2), pages 233-237, May.
    2. Joshi, Sumit, 2003. "The stochastic turnpike property without uniformity in convex aggregate growth models," Journal of Economic Dynamics and Control, Elsevier, vol. 27(7), pages 1289-1315, May.
    3. Paul A. Samuelson & Robert M. Solow, 1956. "A Complete Capital Model Involving Heterogeneous Capital Goods," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 70(4), pages 537-562.
    4. Cox, John C. & Huang, Chi-fu, 1992. "A continuous-time portfolio turnpike theorem," Journal of Economic Dynamics and Control, Elsevier, vol. 16(3-4), pages 491-507.
    5. David Gale, 1967. "On Optimal Development in a Multi-Sector Economy," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 34(1), pages 1-18.
    6. William A. Brock & Leonard J. Mirman, 2001. "Optimal Economic Growth And Uncertainty: The Discounted Case," Chapters, in: W. D. Dechert (ed.), Growth Theory, Nonlinear Dynamics and Economic Modelling, chapter 1, pages 3-37, Edward Elgar Publishing.
    7. M. Marena & L. Montrucchio, 1999. "Neighborhood Turnpike Theorem for Continuous-Time Optimization Models," Journal of Optimization Theory and Applications, Springer, vol. 101(3), pages 651-676, June.
    8. Cuong Le Van & Lisa Morhaim, 2006. "On optimal growth models when the discount factor is near 1 or equal to 1," International Journal of Economic Theory, The International Society for Economic Theory, vol. 2(1), pages 55-76, March.
    9. Takashi Kamihigashi & Santanu Roy, 2006. "Dynamic optimization with a nonsmooth, nonconvex technology: the case of a linear objective function," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 29(2), pages 325-340, October.
    10. Araujo, A & Scheinkman, Jose A, 1977. "Smoothness, Comparative Dynamics, and the Turnpike Property," Econometrica, Econometric Society, vol. 45(3), pages 601-620, April.
    11. Lionel W. McKenzie, 2012. "turnpike theory," The New Palgrave Dictionary of Economics,, Palgrave Macmillan.
    12. Lionel W. McKenzie, 1963. "The Turnpike Theorem of Morishima," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 30(3), pages 169-176.
    13. Mirman, Leonard J. & Zilcha, Itzhak, 1975. "On optimal growth under uncertainty," Journal of Economic Theory, Elsevier, vol. 11(3), pages 329-339, December.
    14. M. Morishima, 1961. "Prices and the Turnpike: II. Proof of a Turnpike Theorem: The "No Joint Production" Case," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 28(2), pages 89-97.
    15. Joshi, Sumit, 1997. "Turnpike Theorems in Nonconvex Nonstationary Environments," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 38(1), pages 225-248, February.
    16. Guerrero-Luchtenberg, C.L., 2000. "A uniform neighborhood turnpike theorem and applications," Journal of Mathematical Economics, Elsevier, vol. 34(3), pages 329-357, November.
    17. Fershtman, Chaim & Kamien, Morton I, 1990. "Turnpike Properties in a Finite-Horizon Differential Game: Dynamic Duopoly with Sticky Prices," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 31(1), pages 49-60, February.
    18. Majumdar, Mukul & Mitra, Tapan, 1982. "Intertemporal allocation with a non-convex technology: The aggregative framework," Journal of Economic Theory, Elsevier, vol. 27(1), pages 101-136, June.
    19. Huberman, Gur & Ross, Stephen, 1983. "Portfolio Turnpike Theorems, Risk Aversion, and Regularly Varying Utility Functions," Econometrica, Econometric Society, vol. 51(5), pages 1345-1361, September.
    20. Feinstein, C. D. & Oren, S. S., 1985. "A funnel turnpike theorem for optimal growth problems with discounting," Journal of Economic Dynamics and Control, Elsevier, vol. 9(1), pages 25-39, September.
    21. Alexandre Scheinkman, Jose, 1976. "On optimal steady states of n-sector growth models when utility is discounted," Journal of Economic Theory, Elsevier, vol. 12(1), pages 11-30, February.
    22. Chang, Fwu-Ranq, 1982. "A note on the stochastic value loss assumption," Journal of Economic Theory, Elsevier, vol. 26(1), pages 164-170, February.
    23. Dybvig, Philip H & Rogers, L C G & Back, Kerry, 1999. "Portfolio Turnpikes," The Review of Financial Studies, Society for Financial Studies, vol. 12(1), pages 165-195.
    24. Andrzej S. Nowak, 1999. "Sensitive equilibria for ergodic stochastic games with countable state spaces," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 50(1), pages 65-76, August.
    25. Coles, Jeffrey Link, 1985. "Equilibrium Turnpike Theory with Constant Returns to Scale and Possible Heterogeneous Discount Factors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 26(3), pages 671-679, October.
    26. Dasgupta, S. & McKenzie, Lionel W., 1985. "A note on comparative statics and dynamics of stationary states," Economics Letters, Elsevier, vol. 18(4), pages 333-338.
    27. Joël Blot & Bertrand Crettez, 2007. "On the smoothness of optimal paths II: some local turnpike results," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 30(2), pages 137-150, November.
    28. Montrucchio, Luigi, 1995. "A New Turnpike Theorem for Discounted Programs," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 5(3), pages 371-382, May.
    29. Brock, W. A. & Majumdar, M., 1978. "Global asymptotic stability results for multisector models of optimal growth under uncertainty when future utilities are discounted," Journal of Economic Theory, Elsevier, vol. 18(2), pages 225-243, August.
    30. A. Haurie & Christian van Delft, 1991. "Turnpike properties for a class of piecewise deterministic systems arising in manufacturing flow control," Post-Print hal-00482416, HAL.
    31. Cuong Le Van & Lisa Morhaim, 2006. "On optimal growth models when the discount factor is near 1 or equal to 1," Post-Print halshs-00096034, HAL.
    32. Haurie, A., 1976. "Optimal control on an infinite time horizon : The turnpike approach," Journal of Mathematical Economics, Elsevier, vol. 3(1), pages 81-102, March.
    33. McKenzie, Lionel W, 1998. "Turnpikes," American Economic Review, American Economic Association, vol. 88(2), pages 1-14, May.
    34. Hiroshi Atsumi, 1965. "Neoclassical Growth and the Efficient Program of Capital Accumulation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 32(2), pages 127-136.
    35. Yano, Makoto, 1984. "Competitive Equilibria on Turnpikes in a McKenzie Economy, I: A Neighborhood Turnpike Theorem," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 25(3), pages 695-717, October.
    36. Majumdar, Mukul & Zilcha, Itzhak, 1987. "Optimal growth in a stochastic environment: Some sensitivity and turnpike results," Journal of Economic Theory, Elsevier, vol. 43(1), pages 116-133, October.
    37. Mirman, Leonard J. & Zilcha, Itzhak, 1977. "Characterizing optimal policies in a one-sector model of economic growth under uncertainty," Journal of Economic Theory, Elsevier, vol. 14(2), pages 389-401, April.
    38. Bewley, Truman, 1982. "An integration of equilibrium theory and turnpike theory," Journal of Mathematical Economics, Elsevier, vol. 10(2-3), pages 233-267, September.
    39. Montrucchio, Luigi, 1995. "A turnpike theorem for continuous-time optimal-control models," Journal of Economic Dynamics and Control, Elsevier, vol. 19(3), pages 599-619, April.
    40. Vassili N Kolokoltsov & Oleg A Malafeyev, 2010. "Understanding Game Theory:Introduction to the Analysis of Many Agent Systems with Competition and Cooperation," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 7564, February.
    41. Hyun Park, 2000. "Global asymptotic stability of a competitive equilibrium with recursive preferences," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 15(3), pages 565-584.
    42. Musa A. Mamedov, 2009. "Asymptotical stability of optimal paths in nonconvex problems," Springer Optimization and Its Applications, in: Charles Pearce & Emma Hunt (ed.), Optimization, edition 1, chapter 0, pages 95-134, Springer.
    43. Brock, William A & Mirman, Leonard J, 1973. "Optimal Economic Growth and Uncertainty: The No Discounting Case," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 14(3), pages 560-573, October.
    44. Roy Radner, 1961. "Prices and the Turnpike: III. Paths of Economic Growth that are Optimal with Regard only to Final States: A Turnpike Theorem," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 28(2), pages 98-104.
    45. Jeremy F. Shapiro, 1968. "Turnpike Planning Horizons for a Markovian Decision Model," Management Science, INFORMS, vol. 14(5), pages 292-300, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lilia Maliar & Serguei Maliar, 2016. "Ruling Out Multiplicity of Smooth Equilibria in Dynamic Games: A Hyperbolic Discounting Example," Dynamic Games and Applications, Springer, vol. 6(2), pages 243-261, June.
    2. Vassili Kolokoltsov, 2017. "The Evolutionary Game of Pressure (or Interference), Resistance and Collaboration," Mathematics of Operations Research, INFORMS, vol. 42(4), pages 915-944, November.
    3. V. N. Kolokoltsov & O. A. Malafeyev, 2018. "Corruption and botnet defense: a mean field game approach," International Journal of Game Theory, Springer;Game Theory Society, vol. 47(3), pages 977-999, September.
    4. Andrzej Nowak & Eilon Solan & Sylvain Sorin, 2013. "Preface: Special Issue on Stochastic Games," Dynamic Games and Applications, Springer, vol. 3(2), pages 125-127, June.
    5. Vassili N. Kolokoltsov, 2022. "Dynamic Quantum Games," Dynamic Games and Applications, Springer, vol. 12(2), pages 552-573, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lars J. Olson & Santanu Roy, 2006. "Theory of Stochastic Optimal Economic Growth," Springer Books, in: Rose-Anne Dana & Cuong Le Van & Tapan Mitra & Kazuo Nishimura (ed.), Handbook on Optimal Growth 1, chapter 11, pages 297-335, Springer.
    2. Joshi, Sumit, 2003. "The stochastic turnpike property without uniformity in convex aggregate growth models," Journal of Economic Dynamics and Control, Elsevier, vol. 27(7), pages 1289-1315, May.
    3. Dai, Darong, 2011. "Modeling the minimum time needed to economic maturity," MPRA Paper 40583, University Library of Munich, Germany, revised 08 Aug 2012.
    4. Darong Dai, 2013. "Wealth Martingale and Neighborhood Turnpike Property In Dynamically Complete Market With Heterogeneous Investors," Economic Research Guardian, Weissberg Publishing, vol. 3(2), pages 86-110, December.
    5. McKenzie, L., 1999. "The First Conferences on the Theory of Economic Growth," RCER Working Papers 459, University of Rochester - Center for Economic Research (RCER).
    6. Jensen, Martin Kaae, 2012. "Global stability and the “turnpike” in optimal unbounded growth models," Journal of Economic Theory, Elsevier, vol. 147(2), pages 802-832.
    7. Darong Dai, 2012. "Stochastic Versions of Turnpike Theorems in the Sense of Uniform Topology," Annals of Economics and Finance, Society for AEF, vol. 13(2), pages 381-423, November.
    8. Dai, Darong, 2011. "Wealth Martingale and Neighborhood Turnpike Property in Dynamically Complete Market with Heterogeneous Investors," MPRA Paper 46416, University Library of Munich, Germany.
    9. Lilia Maliar & Serguei Maliar & John B. Taylor & Inna Tsener, 2020. "A tractable framework for analyzing a class of nonstationary Markov models," Quantitative Economics, Econometric Society, vol. 11(4), pages 1289-1323, November.
    10. Serguei Maliar & John Taylor & Lilia Maliar, 2016. "The Impact of Alternative Transitions to Normalized Monetary Policy," 2016 Meeting Papers 794, Society for Economic Dynamics.
    11. Kamihigashi, Takashi & Roy, Santanu, 2007. "A nonsmooth, nonconvex model of optimal growth," Journal of Economic Theory, Elsevier, vol. 132(1), pages 435-460, January.
    12. M. Khan & Alexander Zaslavski, 2007. "On a Uniform Turnpike of the Third Kind in the Robinson-Solow-Srinivasan Model," Journal of Economics, Springer, vol. 92(2), pages 137-166, October.
    13. Dai, Darong, 2012. "A Robust Turnpike Deduced by Economic Maturity," MPRA Paper 48818, University Library of Munich, Germany.
    14. Dai, Darong, 2011. "Modeling the minimum time needed to economic maturity," MPRA Paper 40386, University Library of Munich, Germany, revised 31 Jul 2012.
    15. Dai, Darong, 2011. "Stochastic Versions of Turnpike Theorems in the Sense of Uniform Topology," MPRA Paper 40216, University Library of Munich, Germany.
    16. Wolff, Reiner, 1997. "Saddle-point dynamics in non-autonomous models of multisector growth with variable returns to scale," Journal of Mathematical Economics, Elsevier, vol. 27(3), pages 267-282, April.
    17. Truman Bewley, 2010. "An Integration of Equilibrium Theory and Turnpike Theory," Levine's Working Paper Archive 1381, David K. Levine.
    18. Yano, Makoto, 1984. "The turnpike of dynamic general equilibrium paths and its insensitivity to initial conditions," Journal of Mathematical Economics, Elsevier, vol. 13(3), pages 235-254, December.
    19. Atsumasa Kondo, 2008. "On The Inefficacy Of Temporary Policy In A Dynamic General Equilibrium With Money," The Japanese Economic Review, Japanese Economic Association, vol. 59(3), pages 324-344, September.
    20. Tapan Mitra & Santanu Roy, 2023. "Stochastic growth, conservation of capital and convergence to a positive steady state," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 76(1), pages 311-351, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:dyngam:v:2:y:2012:i:3:p:294-312. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.