IDEAS home Printed from https://ideas.repec.org/p/ivi/wpasad/2011-21.html
   My bibliography  Save this paper

A reinforcement learning approach to solving incomplete market models with aggregate uncertainty

Author

Listed:
  • Andrei Jirnyi

    (Kellogg School of Management)

  • Vadym Lepetyuk

    (Universidad de Alicante)

Abstract

We develop a method of solving heterogeneous agent models in which individual decisions depend on the entire cross-sectional distribution of individual state variables, such as incomplete market models with liquidity constraints. Our method is based on the principle of reinforcement learning, and does not require parametric assumptions on either the agents' information set, or on the functional form of the aggregate dynamics.

Suggested Citation

  • Andrei Jirnyi & Vadym Lepetyuk, 2011. "A reinforcement learning approach to solving incomplete market models with aggregate uncertainty," Working Papers. Serie AD 2011-21, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    • Handle: RePEc:ivi:wpasad:2011-21
    as

    Download full text from publisher

    File URL: http://www.ivie.es/downloads/docs/wpasad/wpasad-2011-21.pdf
    File Function: Fisrt version / Primera version, 2011
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hugo P. Simão & Jeff Day & Abraham P. George & Ted Gifford & John Nienow & Warren B. Powell, 2009. "An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application," Transportation Science, INFORMS, vol. 43(2), pages 178-197, May.
    2. Maliar, Serguei & Maliar, Lilia & Judd, Kenneth, 2011. "Solving the multi-country real business cycle model using ergodic set methods," Journal of Economic Dynamics and Control, Elsevier, vol. 35(2), pages 207-228, February.
    3. Kim, Sunghyun Henry & Kollmann, Robert & Kim, Jinill, 2010. "Solving the incomplete market model with aggregate uncertainty using a perturbation method," Journal of Economic Dynamics and Control, Elsevier, vol. 34(1), pages 50-58, January.
    4. Maliar, Lilia & Maliar, Serguei & Valli, Fernando, 2010. "Solving the incomplete markets model with aggregate uncertainty using the Krusell-Smith algorithm," Journal of Economic Dynamics and Control, Elsevier, vol. 34(1), pages 42-49, January.
    5. Algan, Yann & Allais, Olivier & Den Haan, Wouter J., 2010. "Solving the incomplete markets model with aggregate uncertainty using parameterized cross-sectional distributions," Journal of Economic Dynamics and Control, Elsevier, vol. 34(1), pages 59-68, January.
    6. Den Haan, Wouter J. & Judd, Kenneth L. & Juillard, Michel, 2011. "Computational suite of models with heterogeneous agents II: Multi-country real business cycle models," Journal of Economic Dynamics and Control, Elsevier, vol. 35(2), pages 175-177, February.
    7. Alan P. Kirman, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-136, Spring.
    8. repec:hal:spmain:info:hdl:2441/72lkhuq5cm8hqrn860asm92bvs is not listed on IDEAS
    9. Reiter, Michael, 2010. "Solving the incomplete markets model with aggregate uncertainty by backward induction," Journal of Economic Dynamics and Control, Elsevier, vol. 34(1), pages 28-35, January.
    10. Per Krusell & Anthony A. Smith & Jr., 1998. "Income and Wealth Heterogeneity in the Macroeconomy," Journal of Political Economy, University of Chicago Press, vol. 106(5), pages 867-896, October.
    11. Juliana Nascimento & Warren Powell, 2010. "Dynamic Programming Models and Algorithms for the Mutual Fund Cash Balance Problem," Management Science, INFORMS, vol. 56(5), pages 801-815, May.
    12. Judd, Kenneth L., 1992. "Projection methods for solving aggregate growth models," Journal of Economic Theory, Elsevier, vol. 58(2), pages 410-452, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shaw, Philip, 2014. "A nonparametric approach to solving a simple one-sector stochastic growth model," Economics Letters, Elsevier, vol. 125(3), pages 447-450.
    2. Maliar, Lilia & Maliar, Serguei & Winant, Pablo, 2021. "Deep learning for solving dynamic economic models," Journal of Monetary Economics, Elsevier, vol. 122(C), pages 76-101.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Takeki Sunakawa, 2020. "Applying the Explicit Aggregation Algorithm to Heterogeneous Macro Models," Computational Economics, Springer;Society for Computational Economics, vol. 55(3), pages 845-874, March.
    2. Karsten O. Chipeniuk, 2020. "Optimal Grid Selection for the Numerical Solution of Dynamic Stochastic Optimization Problems," Computational Economics, Springer;Society for Computational Economics, vol. 56(4), pages 883-928, December.
    3. Pichler, Paul, 2011. "Solving the multi-country Real Business Cycle model using a monomial rule Galerkin method," Journal of Economic Dynamics and Control, Elsevier, vol. 35(2), pages 240-251, February.
    4. Chipeniuk, Karsten O. & Katz, Nets Hawk & Walker, Todd B., 2022. "Households, auctioneers, and aggregation," European Economic Review, Elsevier, vol. 141(C).
    5. Jesús Fernández‐Villaverde & Samuel Hurtado & Galo Nuño, 2023. "Financial Frictions and the Wealth Distribution," Econometrica, Econometric Society, vol. 91(3), pages 869-901, May.
    6. Hull, Isaiah, 2015. "Approximate dynamic programming with post-decision states as a solution method for dynamic economic models," Journal of Economic Dynamics and Control, Elsevier, vol. 55(C), pages 57-70.
    7. Senbeta, Sisay, 2011. "How applicable are the new keynesian DSGE models to a typical low-income economy?," MPRA Paper 30931, University Library of Munich, Germany.
    8. Michael C. Hatcher & Eric M. Scheffel, 2016. "Solving the Incomplete Markets Model in Parallel Using GPU Computing and the Krusell–Smith Algorithm," Computational Economics, Springer;Society for Computational Economics, vol. 48(4), pages 569-591, December.
    9. Den Haan, Wouter J. & Rendahl, Pontus, 2010. "Solving the incomplete markets model with aggregate uncertainty using explicit aggregation," Journal of Economic Dynamics and Control, Elsevier, vol. 34(1), pages 69-78, January.
    10. Kollmann, Robert & Maliar, Serguei & Malin, Benjamin A. & Pichler, Paul, 2011. "Comparison of solutions to the multi-country Real Business Cycle model," Journal of Economic Dynamics and Control, Elsevier, vol. 35(2), pages 186-202, February.
    11. Grey Gordon, 2020. "Computing Dynamic Heterogeneous-Agent Economies: Tracking the Distribution," Economic Quarterly, Federal Reserve Bank of Richmond, issue 2Q, pages 61-95.
    12. Judd, Kenneth L. & Maliar, Lilia & Maliar, Serguei & Valero, Rafael, 2014. "Smolyak method for solving dynamic economic models: Lagrange interpolation, anisotropic grid and adaptive domain," Journal of Economic Dynamics and Control, Elsevier, vol. 44(C), pages 92-123.
    13. Den Haan, Wouter J., 2010. "Comparison of solutions to the incomplete markets model with aggregate uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 34(1), pages 4-27, January.
    14. Marlon Azinovic & Luca Gaegauf & Simon Scheidegger, 2022. "Deep Equilibrium Nets," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 63(4), pages 1471-1525, November.
    15. Giusto, Andrea, 2014. "Adaptive learning and distributional dynamics in an incomplete markets model," Journal of Economic Dynamics and Control, Elsevier, vol. 40(C), pages 317-333.
    16. Fernández-Villaverde, J. & Rubio-Ramírez, J.F. & Schorfheide, F., 2016. "Solution and Estimation Methods for DSGE Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 527-724, Elsevier.
    17. Maliar, Lilia & Maliar, Serguei & Winant, Pablo, 2021. "Deep learning for solving dynamic economic models," Journal of Monetary Economics, Elsevier, vol. 122(C), pages 76-101.
    18. Kenneth L. Judd & Lilia Maliar & Serguei Maliar, 2010. "A Cluster-Grid Projection Method: Solving Problems with High Dimensionality," NBER Working Papers 15965, National Bureau of Economic Research, Inc.
    19. Emoto, Masakazu & Sunakawa, Takeki, 2021. "Applying the explicit aggregation algorithm to heterogeneous agent models in continuous time," Economics Letters, Elsevier, vol. 206(C).
    20. Christian Bayer & Ralph Luetticke, 2020. "Solving discrete time heterogeneous agent models with aggregate risk and many idiosyncratic states by perturbation," Quantitative Economics, Econometric Society, vol. 11(4), pages 1253-1288, November.

    More about this item

    Keywords

    Heterogeneous agents; macroeconomics; dynamic programming; reinforcement learning.;
    All these keywords.

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • C68 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computable General Equilibrium Models
    • E20 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - General (includes Measurement and Data)

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ivi:wpasad:2011-21. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Departamento de Edición (email available below). General contact details of provider: https://edirc.repec.org/data/ievages.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.