IDEAS home Printed from https://ideas.repec.org/a/spr/pardea/v1y2020i2d10.1007_s42985-019-0006-9.html
   My bibliography  Save this article

A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations

Author

Listed:
  • Martin Hutzenthaler

    (University of Duisburg-Essen)

  • Arnulf Jentzen

    (ETH Zurich
    University of Münster)

  • Thomas Kruse

    (University of Gießen)

  • Tuan Anh Nguyen

    (University of Duisburg-Essen)

Abstract

Deep neural networks and other deep learning methods have very successfully been applied to the numerical approximation of high-dimensional nonlinear parabolic partial differential equations (PDEs), which are widely used in finance, engineering, and natural sciences. In particular, simulations indicate that algorithms based on deep learning overcome the curse of dimensionality in the numerical approximation of solutions of semilinear PDEs. For certain linear PDEs it has also been proved mathematically that deep neural networks overcome the curse of dimensionality in the numerical approximation of solutions of such linear PDEs. The key contribution of this article is to rigorously prove this for the first time for a class of nonlinear PDEs. More precisely, we prove in the case of semilinear heat equations with gradient-independent nonlinearities that the numbers of parameters of the employed deep neural networks grow at most polynomially in both the PDE dimension and the reciprocal of the prescribed approximation accuracy. Our proof relies on recently introduced full history recursive multilevel Picard approximations for semilinear PDEs.

Suggested Citation

  • Martin Hutzenthaler & Arnulf Jentzen & Thomas Kruse & Tuan Anh Nguyen, 2020. "A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations," Partial Differential Equations and Applications, Springer, vol. 1(2), pages 1-34, April.
    • Handle: RePEc:spr:pardea:v:1:y:2020:i:2:d:10.1007_s42985-019-0006-9
    DOI: 10.1007/s42985-019-0006-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s42985-019-0006-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s42985-019-0006-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Masaaki Fujii & Akihiko Takahashi & Masayuki Takahashi, 2017. "Asymptotic Expansion as Prior Knowledge in Deep Learning Method for high dimensional BSDEs," CIRJE F-Series CIRJE-F-1069, CIRJE, Faculty of Economics, University of Tokyo.
    2. Justin Sirignano & Konstantinos Spiliopoulos, 2017. "DGM: A deep learning algorithm for solving partial differential equations," Papers 1708.07469, arXiv.org, revised Sep 2018.
    3. Masaaki Fujii & Akihiko Takahashi & Masayuki Takahashi, 2017. "Asymptotic Expansion as Prior Knowledge in Deep Learning Method for high dimensional BSDEs," CARF F-Series CARF-F-423, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Glau, Kathrin & Wunderlich, Linus, 2022. "The deep parametric PDE method and applications to option pricing," Applied Mathematics and Computation, Elsevier, vol. 432(C).
    2. Akihiko Takahashi & Toshihiro Yamada, 2023. "Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus," CIRJE F-Series CIRJE-F-1212, CIRJE, Faculty of Economics, University of Tokyo.
    3. Rawin Assabumrungrat & Kentaro Minami & Masanori Hirano, 2023. "Error Analysis of Option Pricing via Deep PDE Solvers: Empirical Study," Papers 2311.07231, arXiv.org.
    4. Lukas Gonon, 2022. "Deep neural network expressivity for optimal stopping problems," Papers 2210.10443, arXiv.org.
    5. Lukas Gonon, 2024. "Deep neural network expressivity for optimal stopping problems," Finance and Stochastics, Springer, vol. 28(3), pages 865-910, July.
    6. Jentzen, Arnulf & Welti, Timo, 2023. "Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation," Applied Mathematics and Computation, Elsevier, vol. 455(C).
    7. Akihiko Takahashi & Toshihiro Yamada, 2023. "Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus (Forthcoming in "Partial Differential Equations and Applications&quo," CARF F-Series CARF-F-560, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    8. Fred Espen Benth & Nils Detering & Luca Galimberti, 2022. "Pricing options on flow forwards by neural networks in Hilbert space," Papers 2202.11606, arXiv.org.
    9. Antoine Jacquier & Zan Zuric, 2023. "Random neural networks for rough volatility," Papers 2305.01035, arXiv.org.
    10. Ariel Neufeld & Philipp Schmocker & Sizhou Wu, 2024. "Full error analysis of the random deep splitting method for nonlinear parabolic PDEs and PIDEs," Papers 2405.05192, arXiv.org, revised Sep 2024.
    11. Beatrice Acciaio & Anastasis Kratsios & Gudmund Pammer, 2022. "Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer," Papers 2201.13094, arXiv.org, revised Mar 2023.
    12. Maximilien Germain & Mathieu Laurière & Huyên Pham & Xavier Warin, 2022. "DeepSets and their derivative networks for solving symmetric PDEs ," Post-Print hal-03154116, HAL.
    13. Li, Wei & Zhang, Ying & Huang, Dongmei & Rajic, Vesna, 2022. "Study on stationary probability density of a stochastic tumor-immune model with simulation by ANN algorithm," Chaos, Solitons & Fractals, Elsevier, vol. 159(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christian Beck & Lukas Gonon & Arnulf Jentzen, 2024. "Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations," Partial Differential Equations and Applications, Springer, vol. 5(6), pages 1-47, December.
    2. Andrew Na & Justin Wan, 2023. "Efficient Pricing and Hedging of High Dimensional American Options Using Recurrent Networks," Papers 2301.08232, arXiv.org.
    3. Yangang Chen & Justin W. L. Wan, 2019. "Deep Neural Network Framework Based on Backward Stochastic Differential Equations for Pricing and Hedging American Options in High Dimensions," Papers 1909.11532, arXiv.org.
    4. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "Bitcoin technical trading with artificial neural network," CIRJE F-Series CIRJE-F-1078, CIRJE, Faculty of Economics, University of Tokyo.
    5. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "Bitcoin technical trading with artificial neural network," CARF F-Series CARF-F-430, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    6. Nakano, Masafumi & Takahashi, Akihiko & Takahashi, Soichiro, 2018. "Bitcoin technical trading with artificial neural network," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 510(C), pages 587-609.
    7. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "Bitcoin Technical Trading with Articial Neural Network," CIRJE F-Series CIRJE-F-1090, CIRJE, Faculty of Economics, University of Tokyo.
    8. Warin Xavier, 2018. "Nesting Monte Carlo for high-dimensional non-linear PDEs," Monte Carlo Methods and Applications, De Gruyter, vol. 24(4), pages 225-247, December.
    9. Philipp Grohs & Fabian Hornung & Arnulf Jentzen & Philippe von Wurstemberger, 2018. "A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations," Papers 1809.02362, arXiv.org, revised Jan 2023.
    10. Kristina O. F. Williams & Benjamin F. Akers, 2023. "Numerical Simulation of the Korteweg–de Vries Equation with Machine Learning," Mathematics, MDPI, vol. 11(13), pages 1-14, June.
    11. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Working Papers hal-03145949, HAL.
    12. Zhouzhou Gu & Mathieu Lauri`ere & Sebastian Merkel & Jonathan Payne, 2024. "Global Solutions to Master Equations for Continuous Time Heterogeneous Agent Macroeconomic Models," Papers 2406.13726, arXiv.org.
    13. Parand, K. & Aghaei, A.A. & Jani, M. & Ghodsi, A., 2021. "A new approach to the numerical solution of Fredholm integral equations using least squares-support vector regression," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 180(C), pages 114-128.
    14. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Papers 2102.09851, arXiv.org, revised Feb 2021.
    15. A. Max Reppen & H. Mete Soner & Valentin Tissot-Daguette, 2022. "Deep Stochastic Optimization in Finance," Papers 2205.04604, arXiv.org.
    16. Sebastian Jaimungal, 2022. "Reinforcement learning and stochastic optimisation," Finance and Stochastics, Springer, vol. 26(1), pages 103-129, January.
    17. Shuaiqiang Liu & Cornelis W. Oosterlee & Sander M. Bohte, 2019. "Pricing Options and Computing Implied Volatilities using Neural Networks," Risks, MDPI, vol. 7(1), pages 1-22, February.
    18. Bastien Baldacci & Joffrey Derchu & Iuliia Manziuk, 2020. "An approximate solution for options market-making in high dimension," Papers 2009.00907, arXiv.org.
    19. Alexandre Pannier & Cristopher Salvi, 2024. "A path-dependent PDE solver based on signature kernels," Papers 2403.11738, arXiv.org, revised Oct 2024.
    20. Rong Du & Duy-Minh Dang, 2023. "Fourier Neural Network Approximation of Transition Densities in Finance," Papers 2309.03966, arXiv.org, revised Sep 2024.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:pardea:v:1:y:2020:i:2:d:10.1007_s42985-019-0006-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.