IDEAS home Printed from https://ideas.repec.org/p/sce/scecf0/235.html
   My bibliography  Save this paper

An Analytical Method To Calculate The Ergodic And Difference Matrices Of The Discounted Markov Decision Processes

Author

Listed:
  • Jan Kaluski

    (The Silesian Technical University)

Abstract

In the paper, a theorem about the existence of the ergodic and difference matrices of the finite state discounted Markov decision processes had been formulated and proved. On the basis of this theorem an analytical method to calculate these matrices is presented. The theorem allows the distribution of the overall value into two parts: the so-called "constant" part, which represents a part of the value related to the ergodic matrix and the "variable" part which represents a sum of the difference matrices. On the basis of the mentioned analytical method a new performance index to the discounted optimal control Markov problem is proposed and some interpretation of the received results is given. The proposed new performance index is formulated as a quotient of the distinguished parts of the overall value. The method is illustrated by two simple examples.

Suggested Citation

  • Jan Kaluski, 2000. "An Analytical Method To Calculate The Ergodic And Difference Matrices Of The Discounted Markov Decision Processes," Computing in Economics and Finance 2000 235, Society for Computational Economics.
    • Handle: RePEc:sce:scecf0:235
    as

    Download full text from publisher

    File URL: http://fmwww.bc.edu/cef00/papers/paper235.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mertens, Jean-Francois, 2002. "Stochastic games," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 47, pages 1809-1832, Elsevier.
    2. Douglas J. White, 1985. "Real Applications of Markov Decision Processes," Interfaces, INFORMS, vol. 15(6), pages 73-83, December.
    3. Chan, M W Luke, 1981. "A Markovian Approach to the Study of the Canadian Cattle Industry," The Review of Economics and Statistics, MIT Press, vol. 63(1), pages 107-116, February.
    4. John R. Hauser & Kenneth J. Wisniewski, 1982. "Dynamic Analysis of Consumer Response to Marketing Strategies," Management Science, INFORMS, vol. 28(5), pages 455-486, May.
    5. Truman Bewley & Elon Kohlberg, 1978. "On Stochastic Games with Stationary Optimal Strategies," Mathematics of Operations Research, INFORMS, vol. 3(2), pages 104-125, May.
    6. Juval Goldwerger, 1977. "Dynamic Programming for a Stochastic Markovian Process with an Application to the Mean Variance Models," Management Science, INFORMS, vol. 23(6), pages 612-620, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dinah Rosenberg & Eilon Solan & Nicolas Vieille, 2003. "The MaxMin value of stochastic games with imperfect monitoring," International Journal of Game Theory, Springer;Game Theory Society, vol. 32(1), pages 133-150, December.
    2. János Flesch & Gijs Schoenmakers & Koos Vrieze, 2009. "Stochastic games on a product state space: the periodic case," International Journal of Game Theory, Springer;Game Theory Society, vol. 38(2), pages 263-289, June.
    3. Abraham Neyman & Sylvain Sorin, 2010. "Repeated games with public uncertain duration process," International Journal of Game Theory, Springer;Game Theory Society, vol. 39(1), pages 29-52, March.
    4. Vieille, Nicolas, 2002. "Stochastic games: Recent results," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 48, pages 1833-1850, Elsevier.
    5. Anna Jaśkiewicz & Andrzej Nowak, 2011. "Stochastic Games with Unbounded Payoffs: Applications to Robust Control in Economics," Dynamic Games and Applications, Springer, vol. 1(2), pages 253-279, June.
    6. John H. Roberts & Charles J. Nelson & Pamela D. Morrison, 2005. "A Prelaunch Diffusion Model for Evaluating Market Defense Strategies," Marketing Science, INFORMS, vol. 24(1), pages 150-164, August.
    7. Abreu, Dilip & Manea, Mihai, 2012. "Markov equilibria in a model of bargaining in networks," Games and Economic Behavior, Elsevier, vol. 75(1), pages 1-16.
    8. Rida Laraki & A.P. Maitra & William Sudderth, 2005. "Two -person zero-sum stochastic games with semicontinuous payoff," Working Papers hal-00243014, HAL.
    9. Casilda Lasso de la Vega & Oscar Volij, 2020. "The value of a draw," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 70(4), pages 1023-1044, November.
    10. Hauser, John R., 2014. "Consideration-set heuristics," Journal of Business Research, Elsevier, vol. 67(8), pages 1688-1699.
    11. VIEILLE, Nicolas & ROSENBERG, Dinah & SOLAN, Eilon, 2002. "Approximating a sequence of observations by a simple process," HEC Research Papers Series 756, HEC Paris.
    12. Eilon Solan & Nicolas Vieille, 2010. "Computing uniformly optimal strategies in two-player stochastic games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 42(1), pages 237-253, January.
    13. Chernonog, Tatyana & Avinadav, Tal, 2016. "A two-state partially observable Markov decision process with three actionsAuthor-Name: Ben-Zvi, Tal," European Journal of Operational Research, Elsevier, vol. 254(3), pages 957-967.
    14. Victoria C. P. Chen & David Ruppert & Christine A. Shoemaker, 1999. "Applying Experimental Design and Regression Splines to High-Dimensional Continuous-State Stochastic Dynamic Programming," Operations Research, INFORMS, vol. 47(1), pages 38-53, February.
    15. VIEILLE, Nicolas, 2001. "Two-player games : a reduction," HEC Research Papers Series 745, HEC Paris.
    16. Guilherme Carmona, 2002. "Monetary trading: an optimal exchange system," Nova SBE Working Paper Series wp420, Universidade Nova de Lisboa, Nova School of Business and Economics.
    17. Frank H. Page & Myrna H. Wooders, 2009. "Endogenous Network Dynamics," Working Papers 2009.28, Fondazione Eni Enrico Mattei.
    18. Miquel Oliu-Barton, 2021. "New Algorithms for Solving Zero-Sum Stochastic Games," Mathematics of Operations Research, INFORMS, vol. 46(1), pages 255-267, February.
    19. Dinah Rosenberg & Eilon Solan & Nicolas Vieille, 2002. "Stochastic Games with a Single Controller and Incomplete Information," Discussion Papers 1346, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    20. Kao, Jih-Forg, 1995. "Optimal recovery strategies for manufacturing systems," European Journal of Operational Research, Elsevier, vol. 80(2), pages 252-263, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sce:scecf0:235. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/sceeeea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.