IDEAS home Printed from https://ideas.repec.org/a/spr/pardea/v3y2022i4d10.1007_s42985-021-00100-z.html
   My bibliography  Save this article

Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms

Author

Listed:
  • Philipp Grohs

    (University of Vienna)

  • Arnulf Jentzen

    (The Chinese University of Hong Kong
    University of Münster
    ETH Zurich)

  • Diyora Salimova

    (ETH Zurich
    ETH Zurich
    Albert-Ludwigs-University of Freiburg)

Abstract

In the past few years deep artificial neural networks (DNNs) have been successfully employed in a large number of computational problems including, e.g., language processing, image recognition, fraud detection, and computational advertisement. Recently, it has also been proposed in the scientific literature to reformulate high-dimensional partial differential equations (PDEs) as stochastic learning problems and to employ DNNs together with stochastic gradient descent methods to approximate the solutions of such high-dimensional PDEs. There are also a few mathematical convergence results in the scientific literature which show that DNNs can approximate solutions of certain PDEs without the curse of dimensionality in the sense that the number of real parameters employed to describe the DNN grows at most polynomially both in the PDE dimension $$d \in {\mathbb {N}}$$ d ∈ N and the reciprocal of the prescribed approximation accuracy $$\varepsilon > 0$$ ε > 0 . One key argument in most of these results is, first, to employ a Monte Carlo approximation scheme which can approximate the solution of the PDE under consideration at a fixed space-time point without the curse of dimensionality and, thereafter, to prove then that DNNs are flexible enough to mimic the behaviour of the employed approximation scheme. Having this in mind, one could aim for a general abstract result which shows under suitable assumptions that if a certain function can be approximated by any kind of (Monte Carlo) approximation scheme without the curse of dimensionality, then the function can also be approximated with DNNs without the curse of dimensionality. It is a subject of this article to make a first step towards this direction. In particular, the main result of this paper, roughly speaking, shows that if a function can be approximated by means of some suitable discrete approximation scheme without the curse of dimensionality and if there exist DNNs which satisfy certain regularity properties and which approximate this discrete approximation scheme without the curse of dimensionality, then the function itself can also be approximated with DNNs without the curse of dimensionality. Moreover, for the number of real parameters used to describe such approximating DNNs we provide an explicit upper bound for the optimal exponent of the dimension $$d \in {\mathbb {N}}$$ d ∈ N of the function under consideration as well as an explicit lower bound for the optimal exponent of the prescribed approximation accuracy $$\varepsilon >0$$ ε > 0 . As an application of this result we derive that solutions of suitable Kolmogorov PDEs can be approximated with DNNs without the curse of dimensionality.

Suggested Citation

  • Philipp Grohs & Arnulf Jentzen & Diyora Salimova, 2022. "Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms," Partial Differential Equations and Applications, Springer, vol. 3(4), pages 1-41, August.
  • Handle: RePEc:spr:pardea:v:3:y:2022:i:4:d:10.1007_s42985-021-00100-z
    DOI: 10.1007/s42985-021-00100-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s42985-021-00100-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s42985-021-00100-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Masaaki Fujii & Akihiko Takahashi & Masayuki Takahashi, 2017. "Asymptotic Expansion as Prior Knowledge in Deep Learning Method for high dimensional BSDEs," Papers 1710.07030, arXiv.org, revised Mar 2019.
    2. Philipp Grohs & Fabian Hornung & Arnulf Jentzen & Philippe von Wurstemberger, 2018. "A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations," Papers 1809.02362, arXiv.org, revised Jan 2023.
    3. Masaaki Fujii & Akihiko Takahashi & Masayuki Takahashi, 2019. "Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDEs," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 26(3), pages 391-408, September.
    4. Sebastian Becker & Patrick Cheridito & Arnulf Jentzen & Timo Welti, 2019. "Solving high-dimensional optimal stopping problems using deep learning," Papers 1908.01602, arXiv.org, revised Aug 2021.
    5. Masaaki Fujii & Akihiko Takahashi & Masayuki Takahashi, 2019. "Asymptotic Expansion as Prior Knowledge in Deep Learning Method for high dimensional BSDEs (Forthcoming in Asia-Pacific Financial Markets)," CARF F-Series CARF-F-456, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    6. Justin Sirignano & Konstantinos Spiliopoulos, 2017. "DGM: A deep learning algorithm for solving partial differential equations," Papers 1708.07469, arXiv.org, revised Sep 2018.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jentzen, Arnulf & Welti, Timo, 2023. "Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation," Applied Mathematics and Computation, Elsevier, vol. 455(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefan Kremsner & Alexander Steinicke & Michaela Szölgyenyi, 2020. "A Deep Neural Network Algorithm for Semilinear Elliptic PDEs with Applications in Insurance Mathematics," Risks, MDPI, vol. 8(4), pages 1-18, December.
    2. Stefan Kremsner & Alexander Steinicke & Michaela Szolgyenyi, 2020. "A deep neural network algorithm for semilinear elliptic PDEs with applications in insurance mathematics," Papers 2010.15757, arXiv.org, revised Dec 2020.
    3. Maximilien Germain & Huyên Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance ," Post-Print hal-03115503, HAL.
    4. Yuga Iguchi & Riu Naito & Yusuke Okano & Akihiko Takahashi & Toshihiro Yamada, 2021. "Deep Asymptotic Expansion with Weak Approximation ," CIRJE F-Series CIRJE-F-1168, CIRJE, Faculty of Economics, University of Tokyo.
    5. Jiefei Yang & Guanglian Li, 2024. "A deep primal-dual BSDE method for optimal stopping problems," Papers 2409.06937, arXiv.org.
    6. Akihiko Takahashi & Yoshifumi Tsuchida & Toshihiro Yamada, 2021. "A New Efficient Approximation Scheme for Solving High-Dimensional Semilinear PDEs: Control Variate Method for Deep BSDE Solver," CIRJE F-Series CIRJE-F-1159, CIRJE, Faculty of Economics, University of Tokyo.
    7. Sebastian Becker & Patrick Cheridito & Arnulf Jentzen & Timo Welti, 2019. "Solving high-dimensional optimal stopping problems using deep learning," Papers 1908.01602, arXiv.org, revised Aug 2021.
    8. Yuga Iguchi & Riu Naito & Yusuke Okano & Akihiko Takahashi & Toshihiro Yamada, 2021. "Deep Asymptotic Expansion: Application to Financial Mathematics(forthcoming in proceedings of IEEE CSDE 2021)," CARF F-Series CARF-F-523, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    9. Yuga Iguchi & Riu Naito & Yusuke Okano & Akihiko Takahashi & Toshihiro Yamada, 2021. "Deep Asymptotic Expansion: Application to Financial Mathematics," CIRJE F-Series CIRJE-F-1178, CIRJE, Faculty of Economics, University of Tokyo.
    10. Choi, So Eun & Jang, Hyun Jin & Lee, Kyungsub & Zheng, Harry, 2021. "Optimal market-Making strategies under synchronised order arrivals with deep neural networks," Journal of Economic Dynamics and Control, Elsevier, vol. 125(C).
    11. Akihiko Takahashi & Yoshifumi Tsuchida & Toshihiro Yamada, 2021. "A new efficient approximation scheme for solving high-dimensional semilinear PDEs: control variate method for Deep BSDE solver," CARF F-Series CARF-F-504, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo, revised Jan 2022.
    12. Akihiko Takahashi & Yoshifumi Tsuchida & Toshihiro Yamada, 2022. "A new efficient approximation scheme for solving high-dimensional semilinear PDEs: control variate method for Deep BSDE solver (Journal of Computational Physics, published online 19 January 2022)," CARF F-Series CARF-F-532, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo, revised Feb 2022.
    13. Akihiko Takahashi & Yoshifumi Tsuchida & Toshihiro Yamada, 2021. "A new efficient approximation scheme for solving high-dimensional semilinear PDEs: control variate method for Deep BSDE solver," Papers 2101.09890, arXiv.org, revised Jan 2021.
    14. Alessandro Gnoatto & Athena Picarelli & Christoph Reisinger, 2020. "Deep xVA solver -- A neural network based counterparty credit risk management framework," Papers 2005.02633, arXiv.org, revised Dec 2022.
    15. Masaaki Fujii, 2019. "Probabilistic Approach to Mean Field Games and Mean Field Type Control Problems with Multiple Populations," CIRJE F-Series CIRJE-F-1133, CIRJE, Faculty of Economics, University of Tokyo.
    16. Lukas Gonon, 2022. "Deep neural network expressivity for optimal stopping problems," Papers 2210.10443, arXiv.org.
    17. Yoshifumi Tsuchida, 2023. "Control Variate Method for Deep BSDE Solver Using Weak Approximation," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 30(2), pages 273-296, June.
    18. Akihiko Takahashi & Toshihiro Yamada, 2021. "Asymptotic Expansion and Deep Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Kolmogorov Partial Differential Equations with Nonlinear Coefficients," CIRJE F-Series CIRJE-F-1167, CIRJE, Faculty of Economics, University of Tokyo.
    19. Christian Beck & Lukas Gonon & Arnulf Jentzen, 2024. "Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations," Partial Differential Equations and Applications, Springer, vol. 5(6), pages 1-47, December.
    20. Masaaki Fujii, 2019. "Probabilistic Approach to Mean Field Games and Mean Field Type Control Problems with Multiple Populations," CARF F-Series CARF-F-467, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.

    More about this item

    Keywords

    65C99; 68T05;

    JEL classification:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:pardea:v:3:y:2022:i:4:d:10.1007_s42985-021-00100-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.