IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2411.08668.html
   My bibliography  Save this paper

A Machine Learning Algorithm for Finite-Horizon Stochastic Control Problems in Economics

Author

Listed:
  • Xianhua Peng
  • Steven Kou
  • Lekang Zhang

Abstract

We propose a machine learning algorithm for solving finite-horizon stochastic control problems based on a deep neural network representation of the optimal policy functions. The algorithm has three features: (1) It can solve high-dimensional (e.g., over 100 dimensions) and finite-horizon time-inhomogeneous stochastic control problems. (2) It has a monotonicity of performance improvement in each iteration, leading to good convergence properties. (3) It does not rely on the Bellman equation. To demonstrate the efficiency of the algorithm, it is applied to solve various finite-horizon time-inhomogeneous problems including recursive utility optimization under a stochastic volatility model, a multi-sector stochastic growth, and optimal control under a dynamic stochastic integration of climate and economy model with eight-dimensional state vectors and 600 time periods.

Suggested Citation

  • Xianhua Peng & Steven Kou & Lekang Zhang, 2024. "A Machine Learning Algorithm for Finite-Horizon Stochastic Control Problems in Economics," Papers 2411.08668, arXiv.org, revised Dec 2024.
    • Handle: RePEc:arx:papers:2411.08668
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2411.08668
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," The Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    2. Yongyang Cai & Thomas S. Lontzek, 2019. "The Social Cost of Carbon with Economic and Climate Risks," Journal of Political Economy, University of Chicago Press, vol. 127(6), pages 2684-2734.
    3. Marlon Azinovic & Jan v{Z}emliv{c}ka, 2023. "Economics-Inspired Neural Networks with Stabilizing Homotopies," Papers 2303.14802, arXiv.org.
    4. George Chacko & Luis M. Viceira, 2005. "Dynamic Consumption and Portfolio Choice with Stochastic Volatility in Incomplete Markets," The Review of Financial Studies, Society for Financial Studies, vol. 18(4), pages 1369-1402.
    5. Victor Duarte & Julia Fonseca & Aaron S. Goodman & Jonathan A. Parker, 2021. "Simple Allocation Rules and Optimal Portfolio Choice Over the Lifecycle," NBER Working Papers 29559, National Bureau of Economic Research, Inc.
    6. Christiano, Lawrence J, 1990. "Linear-Quadratic Approximation and Value-Function Iteration: A Comparison," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 99-113, January.
    7. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, December.
    8. Larry G. Epstein & Stanley E. Zin, 2013. "Substitution, risk aversion and the temporal behavior of consumption and asset returns: A theoretical framework," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 12, pages 207-239, World Scientific Publishing Co. Pte. Ltd..
    9. David B. Brown & James E. Smith, 2014. "Information Relaxations, Duality, and Convex Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 62(6), pages 1394-1415, December.
    10. Lepetyuk, Vadym & Maliar, Lilia & Maliar, Serguei, 2020. "When the U.S. catches a cold, Canada sneezes: A lower-bound tale told by deep learning," Journal of Economic Dynamics and Control, Elsevier, vol. 117(C).
    11. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    12. Schroder, Mark & Skiadas, Costis, 1999. "Optimal Consumption and Portfolio Selection with Stochastic Differential Utility," Journal of Economic Theory, Elsevier, vol. 89(1), pages 68-126, November.
    13. Heston, Steven L, 1993. "A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options," The Review of Financial Studies, Society for Financial Studies, vol. 6(2), pages 327-343.
    14. David B. Brown & James E. Smith & Peng Sun, 2010. "Information Relaxations and Duality in Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 785-801, August.
    15. Lars Peter Hansen & Thomas J. Sargent, 2013. "Recursive Models of Dynamic Linear Economies," Economics Books, Princeton University Press, edition 1, number 10141.
    16. William A. Brock & Leonard J. Mirman, 2001. "Optimal Economic Growth And Uncertainty: The Discounted Case," Chapters, in: W. D. Dechert (ed.), Growth Theory, Nonlinear Dynamics and Economic Modelling, chapter 1, pages 3-37, Edward Elgar Publishing.
    17. repec:dau:papers:123456789/12195 is not listed on IDEAS
    18. Marlon Azinovic & Luca Gaegauf & Simon Scheidegger, 2022. "Deep Equilibrium Nets," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 63(4), pages 1471-1525, November.
    19. Yongyang Cai & Kenneth L. Judd & Thomas S. Lontzek, 2012. "Continuous-Time Methods for Integrated Assessment Models," NBER Working Papers 18365, National Bureau of Economic Research, Inc.
    20. Holger Kraft & Thomas Seiferling & Frank Thomas Seifried, 2017. "Optimal consumption and investment with Epstein–Zin recursive utility," Finance and Stochastics, Springer, vol. 21(1), pages 187-226, January.
    21. Holger Kraft & Frank Seifried & Mogens Steffensen, 2013. "Consumption-portfolio optimization with recursive utility in incomplete markets," Finance and Stochastics, Springer, vol. 17(1), pages 161-196, January.
    22. Broadie, Mark & Glasserman, Paul, 1997. "Pricing American-style securities using simulation," Journal of Economic Dynamics and Control, Elsevier, vol. 21(8-9), pages 1323-1352, June.
    23. Maliar, Lilia & Maliar, Serguei & Winant, Pablo, 2021. "Deep learning for solving dynamic economic models," Journal of Monetary Economics, Elsevier, vol. 122(C), pages 76-101.
    24. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steven Kou & Xianhua Peng & Xingbo Xu, 2016. "EM Algorithm and Stochastic Control in Economics," Papers 1611.01767, arXiv.org.
    2. Fernández-Villaverde, J. & Rubio-Ramírez, J.F. & Schorfheide, F., 2016. "Solution and Estimation Methods for DSGE Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 527-724, Elsevier.
    3. Thomas J. Sargent & John Stachurski, 2024. "Dynamic Programming: Finite States," Papers 2401.10473, arXiv.org.
    4. Victor Duarte & Diogo Duarte & Dejanir H. Silva, 2024. "Machine Learning for Continuous-Time Finance," CESifo Working Paper Series 10909, CESifo.
    5. Lim, Terence & Lo, Andrew W. & Merton, Robert C. & Scholes, Myron S., 2006. "The Derivatives Sourcebook," Foundations and Trends(R) in Finance, now publishers, vol. 1(5–6), pages 365-572, April.
    6. Michael W. Brandt & Amit Goyal & Pedro Santa-Clara & Jonathan R. Stroud, 2005. "A Simulation Approach to Dynamic Portfolio Choice with an Application to Learning About Return Predictability," The Review of Financial Studies, Society for Financial Studies, vol. 18(3), pages 831-873.
    7. Zixin Feng & Dejian Tian, 2021. "Optimal consumption and portfolio selection with Epstein-Zin utility under general constraints," Papers 2111.09032, arXiv.org, revised May 2023.
    8. Suresh M. Sundaresan, 2000. "Continuous‐Time Methods in Finance: A Review and an Assessment," Journal of Finance, American Finance Association, vol. 55(4), pages 1569-1622, August.
    9. Duffie, Darrell, 2003. "Intertemporal asset pricing theory," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, edition 1, volume 1, chapter 11, pages 639-742, Elsevier.
    10. Chen, Xingjiang & Ruan, Xinfeng & Zhang, Wenjun, 2021. "Dynamic portfolio choice and information trading with recursive utility," Economic Modelling, Elsevier, vol. 98(C), pages 154-167.
    11. Benzoni, Luca & Collin-Dufresne, Pierre & Goldstein, Robert S., 2011. "Explaining asset pricing puzzles associated with the 1987 market crash," Journal of Financial Economics, Elsevier, vol. 101(3), pages 552-573, September.
    12. Matoussi, Anis & Xing, Hao, 2018. "Convex duality for Epstein-Zin stochastic differential utility," LSE Research Online Documents on Economics 82519, London School of Economics and Political Science, LSE Library.
    13. Holger Kraft & Thomas Seiferling & Frank Thomas Seifried, 2017. "Optimal consumption and investment with Epstein–Zin recursive utility," Finance and Stochastics, Springer, vol. 21(1), pages 187-226, January.
    14. Immacolata Oliva & Ilaria Stefani, 2023. "Co-jumps and recursive preferences in portfolio choices," Annals of Finance, Springer, vol. 19(3), pages 291-324, September.
    15. Kraft, Holger & Seiferling, Thomas & Seifried, Frank Thomas, 2016. "Optimal consumption and investment with Epstein-Zin recursive utility," SAFE Working Paper Series 52, Leibniz Institute for Financial Research SAFE, revised 2016.
    16. Wei, Pengyu & Yang, Charles & Zhuang, Yi, 2023. "Robust consumption and portfolio choice with derivatives trading," European Journal of Operational Research, Elsevier, vol. 304(2), pages 832-850.
    17. Mark Broadie & Weiwei Shen, 2017. "Numerical solutions to dynamic portfolio problems with upper bounds," Computational Management Science, Springer, vol. 14(2), pages 215-227, April.
    18. Simon Scheidegger & Adrien Treccani, 2021. "Pricing American Options under High-Dimensional Models with Recursive Adaptive Sparse Expectations [Telling from Discrete Data Whether the Underlying Continuous-Time Model Is a Diffusion]," Journal of Financial Econometrics, Oxford University Press, vol. 19(2), pages 258-290.
    19. O. Samimi & Z. Mardani & S. Sharafpour & F. Mehrdoust, 2017. "LSM Algorithm for Pricing American Option Under Heston–Hull–White’s Stochastic Volatility Model," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 173-187, August.
    20. Fahrenwaldt, Matthias Albrecht & Jensen, Ninna Reitzel & Steffensen, Mogens, 2020. "Nonrecursive separation of risk and time preferences," Journal of Mathematical Economics, Elsevier, vol. 90(C), pages 95-108.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2411.08668. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.