IDEAS home Printed from https://ideas.repec.org/a/spr/metcap/v24y2022i1d10.1007_s11009-019-09767-9.html
   My bibliography  Save this article

Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications

Author

Listed:
  • Achref Bachouch

    (Mälardalen University)

  • Côme Huré

    (LPSM, University Paris Diderot)

  • Nicolas Langrené

    (RiskLab Australia)

  • Huyên Pham

    (LPSM, University Paris Diderot
    CREST-ENSAE)

Abstract

This paper presents several numerical applications of deep learning-based algorithms for discrete-time stochastic control problems in finite time horizon that have been introduced in Huré et al. (2018). Numerical and comparative tests using TensorFlow illustrate the performance of our different algorithms, namely control learning by performance iteration (algorithms NNcontPI and ClassifPI), control learning by hybrid iteration (algorithms Hybrid-Now and Hybrid-LaterQ), on the 100-dimensional nonlinear PDEs examples from Weinan et al. (2017) and on quadratic backward stochastic differential equations as in Chassagneux and Richou (2016). We also performed tests on low-dimension control problems such as an option hedging problem in finance, as well as energy storage problems arising in the valuation of gas storage and in microgrid management. Numerical results and comparisons to quantization-type algorithms Qknn, as an efficient algorithm to numerically solve low-dimensional control problems, are also provided.

Suggested Citation

  • Achref Bachouch & Côme Huré & Nicolas Langrené & Huyên Pham, 2022. "Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications," Methodology and Computing in Applied Probability, Springer, vol. 24(1), pages 143-178, March.
    • Handle: RePEc:spr:metcap:v:24:y:2022:i:1:d:10.1007_s11009-019-09767-9
    DOI: 10.1007/s11009-019-09767-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11009-019-09767-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11009-019-09767-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rene Carmona & Michael Ludkovski, 2010. "Valuation of energy storage: an optimal switching approach," Quantitative Finance, Taylor & Francis Journals, vol. 10(4), pages 359-374.
    2. Daniel R. Jiang & Warren B. Powell, 2015. "An Approximate Dynamic Programming Algorithm for Monotone Value Functions," Operations Research, INFORMS, vol. 63(6), pages 1489-1511, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pierre Bras & Gilles Pag`es, 2022. "Langevin algorithms for Markovian Neural Networks and Deep Stochastic control," Papers 2212.12018, arXiv.org, revised Jan 2023.
    2. Alexandre Roch, 2023. "Optimal Liquidation Through a Limit Order Book: A Neural Network and Simulation Approach," Methodology and Computing in Applied Probability, Springer, vol. 25(1), pages 1-29, March.
    3. Pierre Bras & Gilles Pagès, 2022. "Langevin algorithms for Markovian Neural Networks and Deep Stochastic control," Working Papers hal-03980632, HAL.
    4. Olivier Bokanowski & Averil Prost & Xavier Warin, 2023. "Neural networks for first order HJB equations and application to front propagation with obstacle terms," Partial Differential Equations and Applications, Springer, vol. 4(5), pages 1-36, October.
    5. Laurens Van Mieghem & Antonis Papapantoleon & Jonas Papazoglou-Hennig, 2023. "Machine learning for option pricing: an empirical investigation of network architectures," Papers 2307.07657, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Achref Bachouch & C^ome Hur'e & Nicolas Langren'e & Huyen Pham, 2018. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Papers 1812.05916, arXiv.org, revised Jan 2020.
    2. Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
    3. Achref Bachouch & Côme Huré & Nicolas Langrené & Huyen Pham, 2019. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Working Papers hal-01949221, HAL.
    4. Somayeh Moazeni & Warren B. Powell & Boris Defourny & Belgacem Bouzaiene-Ayari, 2017. "Parallel Nonstationary Direct Policy Search for Risk-Averse Stochastic Optimization," INFORMS Journal on Computing, INFORMS, vol. 29(2), pages 332-349, May.
    5. Anton A. Shardin & Michaela Szölgyenyi, 2016. "Optimal Control Of An Energy Storage Facility Under A Changing Economic Environment And Partial Information," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 19(04), pages 1-27, June.
    6. Weitzel, Timm & Glock, Christoph H., 2018. "Energy management for stationary electric energy storage systems: A systematic literature review," European Journal of Operational Research, Elsevier, vol. 264(2), pages 582-606.
    7. Roxana Dumitrescu & Redouane Silvente & Peter Tankov, 2024. "Price impact and long-term profitability of energy storage," Papers 2410.12495, arXiv.org.
    8. Daniel R. Jiang & Warren B. Powell, 2015. "An Approximate Dynamic Programming Algorithm for Monotone Value Functions," Operations Research, INFORMS, vol. 63(6), pages 1489-1511, December.
    9. Hanfeld, Marc & Schlüter, Stephan, 2016. "Operating a swing option on today's gas markets: How least squares Monte Carlo works and why it is beneficial," FAU Discussion Papers in Economics 10/2016, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    10. Nemat Safarov & Colin Atkinson, 2017. "Natural Gas-Fired Power Plants Valuation And Optimization Under Lévy Copulas And Regime Switching," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 20(01), pages 1-38, February.
    11. Cartea, Álvaro & González-Pedraz, Carlos, 2012. "How much should we pay for interconnecting electricity markets? A real options approach," Energy Economics, Elsevier, vol. 34(1), pages 14-30.
    12. Bastian Felix, 2012. "Gas Storage Valuation: A Comparative Simulation Study," EWL Working Papers 1201, University of Duisburg-Essen, Chair for Management Science and Energy Economics, revised Apr 2014.
    13. Alessandro Balata & Michael Ludkovski & Aditya Maheshwari & Jan Palczewski, 2019. "Statistical Learning for Probability-Constrained Stochastic Optimal Control," Papers 1905.00107, arXiv.org, revised Aug 2020.
    14. Andrew J. Collins & Patrick Hester & Barry Ezell & John Horst, 2016. "An improvement selection methodology for key performance indicators," Environment Systems and Decisions, Springer, vol. 36(2), pages 196-208, June.
    15. Felix, Bastian Joachim & Weber, Christoph, 2012. "Gas storage valuation applying numerically constructed recombining trees," European Journal of Operational Research, Elsevier, vol. 216(1), pages 178-187.
    16. Nemat Safarov & Colin Atkinson, 2016. "Natural gas-fired power plants valuation and optimisation under Levy copulas and regime-switching," Papers 1607.01207, arXiv.org, revised Jul 2016.
    17. Achref Bachouch & Côme Huré & Nicolas Langrené & Huyen Pham, 2020. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Post-Print hal-01949221, HAL.
    18. Nicolas Curin & Michael Kettler & Xi Kleisinger-Yu & Vlatka Komaric & Thomas Krabichler & Josef Teichmann & Hanna Wutte, 2021. "A deep learning model for gas storage optimization," Papers 2102.01980, arXiv.org, revised Mar 2021.
    19. Daniel R. Jiang & Warren B. Powell, 2018. "Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 554-579, May.
    20. Lai Wei & Yongpei Guan, 2014. "Optimal Control of Plug-In Hybrid Electric Vehicles with Market Impact and Risk Attitude," Transportation Science, INFORMS, vol. 48(4), pages 467-482, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:metcap:v:24:y:2022:i:1:d:10.1007_s11009-019-09767-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.