IDEAS home Printed from https://ideas.repec.org/p/sce/scecf0/235.html
   My bibliography  Save this paper

An Analytical Method To Calculate The Ergodic And Difference Matrices Of The Discounted Markov Decision Processes

Author

Listed:
  • Jan Kaluski

    (The Silesian Technical University)

Abstract

In the paper, a theorem about the existence of the ergodic and difference matrices of the finite state discounted Markov decision processes had been formulated and proved. On the basis of this theorem an analytical method to calculate these matrices is presented. The theorem allows the distribution of the overall value into two parts: the so-called "constant" part, which represents a part of the value related to the ergodic matrix and the "variable" part which represents a sum of the difference matrices. On the basis of the mentioned analytical method a new performance index to the discounted optimal control Markov problem is proposed and some interpretation of the received results is given. The proposed new performance index is formulated as a quotient of the distinguished parts of the overall value. The method is illustrated by two simple examples.

Suggested Citation

  • Jan Kaluski, 2000. "An Analytical Method To Calculate The Ergodic And Difference Matrices Of The Discounted Markov Decision Processes," Computing in Economics and Finance 2000 235, Society for Computational Economics.
    • Handle: RePEc:sce:scecf0:235
    as

    Download full text from publisher

    File URL: http://fmwww.bc.edu/cef00/papers/paper235.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chan, M W Luke, 1981. "A Markovian Approach to the Study of the Canadian Cattle Industry," The Review of Economics and Statistics, MIT Press, vol. 63(1), pages 107-116, February.
    2. John R. Hauser & Kenneth J. Wisniewski, 1982. "Dynamic Analysis of Consumer Response to Marketing Strategies," Management Science, INFORMS, vol. 28(5), pages 455-486, May.
    3. Mertens, Jean-Francois, 2002. "Stochastic games," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 47, pages 1809-1832, Elsevier.
    4. Truman Bewley & Elon Kohlberg, 1978. "On Stochastic Games with Stationary Optimal Strategies," Mathematics of Operations Research, INFORMS, vol. 3(2), pages 104-125, May.
    5. Juval Goldwerger, 1977. "Dynamic Programming for a Stochastic Markovian Process with an Application to the Mean Variance Models," Management Science, INFORMS, vol. 23(6), pages 612-620, February.
    6. Douglas J. White, 1985. "Real Applications of Markov Decision Processes," Interfaces, INFORMS, vol. 15(6), pages 73-83, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frank H. Page & Myrna H. Wooders, 2009. "Endogenous Network Dynamics," Working Papers 2009.28, Fondazione Eni Enrico Mattei.
    2. Dinah Rosenberg & Eilon Solan & Nicolas Vieille, 2003. "The MaxMin value of stochastic games with imperfect monitoring," International Journal of Game Theory, Springer;Game Theory Society, vol. 32(1), pages 133-150, December.
    3. János Flesch & Gijs Schoenmakers & Koos Vrieze, 2009. "Stochastic games on a product state space: the periodic case," International Journal of Game Theory, Springer;Game Theory Society, vol. 38(2), pages 263-289, June.
    4. Dinah Rosenberg & Eilon Solan & Nicolas Vieille, 1999. "Stopping Games with Randomized Strategies," Discussion Papers 1258, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    5. Abraham Neyman & Sylvain Sorin, 2010. "Repeated games with public uncertain duration process," International Journal of Game Theory, Springer;Game Theory Society, vol. 39(1), pages 29-52, March.
    6. Kim, C.S. & Hallahan, Charles B. & Schaible, Glenn D. & Leath, Mack N., 2000. "A Decomposed Regression Model For Measuring Structural Changes In The Flour Milling Industry," 2000 Annual meeting, July 30-August 2, Tampa, FL 21834, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    7. Vieille, Nicolas, 2002. "Stochastic games: Recent results," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 48, pages 1833-1850, Elsevier.
    8. Anna Jaśkiewicz & Andrzej Nowak, 2011. "Stochastic Games with Unbounded Payoffs: Applications to Robust Control in Economics," Dynamic Games and Applications, Springer, vol. 1(2), pages 253-279, June.
    9. John H. Roberts & Charles J. Nelson & Pamela D. Morrison, 2005. "A Prelaunch Diffusion Model for Evaluating Market Defense Strategies," Marketing Science, INFORMS, vol. 24(1), pages 150-164, August.
    10. Antonello Maruotti & Jan Bulla & Tanya Mark, 2019. "Assessing the influence of marketing activities on customer behaviors: a dynamic clustering approach," METRON, Springer;Sapienza Università di Roma, vol. 77(1), pages 19-42, April.
    11. Chen, Victoria C. P., 1999. "Application of orthogonal arrays and MARS to inventory forecasting stochastic dynamic programs," Computational Statistics & Data Analysis, Elsevier, vol. 30(3), pages 317-341, May.
    12. Abreu, Dilip & Manea, Mihai, 2012. "Markov equilibria in a model of bargaining in networks," Games and Economic Behavior, Elsevier, vol. 75(1), pages 1-16.
    13. Rida Laraki & A.P. Maitra & William Sudderth, 2005. "Two -person zero-sum stochastic games with semicontinuous payoff," Working Papers hal-00243014, HAL.
    14. Casilda Lasso de la Vega & Oscar Volij, 2020. "The value of a draw," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 70(4), pages 1023-1044, November.
    15. Abraham Neyman, 2002. "Stochastic games: Existence of the MinMax," Discussion Paper Series dp295, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
    16. Laraki, Rida & Sorin, Sylvain, 2015. "Advances in Zero-Sum Dynamic Games," Handbook of Game Theory with Economic Applications,, Elsevier.
    17. VIEILLE, Nicolas & ROSENBERG, Dinah & SOLAN, Eilon, 2002. "Stochastic games with a single controller and incomplete information," HEC Research Papers Series 754, HEC Paris.
    18. Levy, Yehuda, 2012. "Stochastic games with information lag," Games and Economic Behavior, Elsevier, vol. 74(1), pages 243-256.
    19. Zong-Zhi Lin & James C. Bean & Chelsea C. White, 2004. "A Hybrid Genetic/Optimization Algorithm for Finite-Horizon, Partially Observed Markov Decision Processes," INFORMS Journal on Computing, INFORMS, vol. 16(1), pages 27-38, February.
    20. Hauser, John R., 2014. "Consideration-set heuristics," Journal of Business Research, Elsevier, vol. 67(8), pages 1688-1699.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sce:scecf0:235. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/sceeeea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.