IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-01949221.html
   My bibliography  Save this paper

Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications

Author

Listed:
  • Achref Bachouch

    (UiO - University of Oslo)

  • Côme Huré

    (LPSM (UMR_8001) - Laboratoire de Probabilités, Statistique et Modélisation - UPD7 - Université Paris Diderot - Paris 7 - SU - Sorbonne Université - CNRS - Centre National de la Recherche Scientifique)

  • Nicolas Langrené

    (CSIRO - Data61 [Canberra] - ANU - Australian National University - CSIRO - Commonwealth Scientific and Industrial Research Organisation [Canberra])

  • Huyen Pham

    (LPSM (UMR_8001) - Laboratoire de Probabilités, Statistique et Modélisation - UPD7 - Université Paris Diderot - Paris 7 - SU - Sorbonne Université - CNRS - Centre National de la Recherche Scientifique)

Abstract

This paper presents several numerical applications of deep learning-based algorithms that have been introduced in [HPBL18]. Numerical and comparative tests using TensorFlow illustrate the performance of our different algorithms, namely control learning by performance iteration (algorithms NNcontPI and ClassifPI), control learning by hybrid iteration (algorithms Hybrid-Now and Hybrid-LaterQ), on the 100-dimensional nonlinear PDEs examples from [EHJ17] and on quadratic backward stochastic differential equations as in [CR16]. We also performed tests on low-dimension control problems such as an option hedging problem in finance, as well as energy storage problems arising in the valuation of gas storage and in microgrid management. Numerical results and comparisons to quantization-type algorithms Qknn, as an efficient algorithm to numerically solve low-dimensional control problems, are also provided; and some corresponding codes are available on https://github.com/comeh/.

Suggested Citation

  • Achref Bachouch & Côme Huré & Nicolas Langrené & Huyen Pham, 2020. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Post-Print hal-01949221, HAL.
    • Handle: RePEc:hal:journl:hal-01949221
    Note: View the original document on HAL open archive server: https://hal.science/hal-01949221v3
    as

    Download full text from publisher

    File URL: https://hal.science/hal-01949221v3/document
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Daniel R. Jiang & Warren B. Powell, 2015. "An Approximate Dynamic Programming Algorithm for Monotone Value Functions," Operations Research, INFORMS, vol. 63(6), pages 1489-1511, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicolas Curin & Michael Kettler & Xi Kleisinger-Yu & Vlatka Komaric & Thomas Krabichler & Josef Teichmann & Hanna Wutte, 2021. "A deep learning model for gas storage optimization," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 44(2), pages 1021-1037, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weitzel, Timm & Glock, Christoph H., 2018. "Energy management for stationary electric energy storage systems: A systematic literature review," European Journal of Operational Research, Elsevier, vol. 264(2), pages 582-606.
    2. Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
    3. Al-Kanj, Lina & Nascimento, Juliana & Powell, Warren B., 2020. "Approximate dynamic programming for planning a ride-hailing system using autonomous fleets of electric vehicles," European Journal of Operational Research, Elsevier, vol. 284(3), pages 1088-1106.
    4. Achref Bachouch & Côme Huré & Nicolas Langrené & Huyên Pham, 2022. "Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications," Methodology and Computing in Applied Probability, Springer, vol. 24(1), pages 143-178, March.
    5. Ulmer, Marlin W. & Thomas, Barrett W., 2020. "Meso-parametric value function approximation for dynamic customer acceptances in delivery routing," European Journal of Operational Research, Elsevier, vol. 285(1), pages 183-195.
    6. Andersen, Jesper Fink & Andersen, Anders Reenberg & Kulahci, Murat & Nielsen, Bo Friis, 2022. "A numerical study of Markov decision process algorithms for multi-component replacement problems," European Journal of Operational Research, Elsevier, vol. 299(3), pages 898-909.
    7. Sebastian Becker & Patrick Cheridito & Arnulf Jentzen & Timo Welti, 2019. "Solving high-dimensional optimal stopping problems using deep learning," Papers 1908.01602, arXiv.org, revised Aug 2021.
    8. Andrew J. Collins & Patrick Hester & Barry Ezell & John Horst, 2016. "An improvement selection methodology for key performance indicators," Environment Systems and Decisions, Springer, vol. 36(2), pages 196-208, June.
    9. Achref Bachouch & C^ome Hur'e & Nicolas Langren'e & Huyen Pham, 2018. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Papers 1812.05916, arXiv.org, revised Jan 2020.
    10. Daniel R. Jiang & Warren B. Powell, 2018. "Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 554-579, May.
    11. Antoine Jacquier & Hao Liu, 2017. "Optimal liquidation in a Level-I limit order book for large tick stocks," Papers 1701.01327, arXiv.org, revised Nov 2017.
    12. Chen, Yao & Liu, Yang & Bai, Yun & Mao, Baohua, 2024. "Real-time dispatch management of shared autonomous vehicles with on-demand and pre-booked requests," Transportation Research Part A: Policy and Practice, Elsevier, vol. 181(C).
    13. Fokkema, Jan Eise & uit het Broek, Michiel A.J. & Schrotenboer, Albert H. & Land, Martin J. & Van Foreest, Nicky D., 2022. "Seasonal hydrogen storage decisions under constrained electricity distribution capacity," Renewable Energy, Elsevier, vol. 195(C), pages 76-91.
    14. Achref Bachouch & Côme Huré & Nicolas Langrené & Huyen Pham, 2019. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Working Papers hal-01949221, HAL.
    15. Yin, Jiateng & Tang, Tao & Yang, Lixing & Gao, Ziyou & Ran, Bin, 2016. "Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 178-210.

    More about this item

    Keywords

    reinforcement learning; Policy iteration algorithm; Deep learning; value iteration; quantization;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-01949221. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.