IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2209.14430.html
   My bibliography  Save this paper

Minimax Optimal Kernel Operator Learning via Multilevel Training

Author

Listed:
  • Jikai Jin
  • Yiping Lu
  • Jose Blanchet
  • Lexing Ying

Abstract

Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional data analysis, causal inference, and multi-agent reinforcement learning. In this paper, we study the statistical limit of learning a Hilbert-Schmidt operator between two infinite-dimensional Sobolev reproducing kernel Hilbert spaces. We establish the information-theoretic lower bound in terms of the Sobolev Hilbert-Schmidt norm and show that a regularization that learns the spectral components below the bias contour and ignores the ones that are above the variance contour can achieve the optimal learning rate. At the same time, the spectral components between the bias and variance contours give us flexibility in designing computationally feasible machine learning algorithms. Based on this observation, we develop a multilevel kernel operator learning algorithm that is optimal when learning linear operators between infinite-dimensional function spaces.

Suggested Citation

  • Jikai Jin & Yiping Lu & Jose Blanchet & Lexing Ying, 2022. "Minimax Optimal Kernel Operator Learning via Multilevel Training," Papers 2209.14430, arXiv.org, revised Jul 2023.
  • Handle: RePEc:arx:papers:2209.14430
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2209.14430
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Michael B. Giles, 2008. "Multilevel Monte Carlo Path Simulation," Operations Research, INFORMS, vol. 56(3), pages 607-617, June.
    2. Reimherr, Matthew, 2015. "Functional regression with repeated eigenvalues," Statistics & Probability Letters, Elsevier, vol. 107(C), pages 62-70.
    3. Justin Sirignano & Konstantinos Spiliopoulos, 2017. "DGM: A deep learning algorithm for solving partial differential equations," Papers 1708.07469, arXiv.org, revised Sep 2018.
    4. Chang-Han Rhee & Peter W. Glynn, 2015. "Unbiased Estimation with Square Root Convergence for SDE Models," Operations Research, INFORMS, vol. 63(5), pages 1026-1043, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wei Fang & Zhenru Wang & Michael B. Giles & Chris H. Jackson & Nicky J. Welton & Christophe Andrieu & Howard Thom, 2022. "Multilevel and Quasi Monte Carlo Methods for the Calculation of the Expected Value of Partial Perfect Information," Medical Decision Making, , vol. 42(2), pages 168-181, February.
    2. Michael B. Giles & Abdul-Lateef Haji-Ali & Jonathan Spence, 2023. "Efficient Risk Estimation for the Credit Valuation Adjustment," Papers 2301.05886, arXiv.org, revised May 2024.
    3. Michael B. Giles & Abdul-Lateef Haji-Ali, 2019. "Sub-sampling and other considerations for efficient risk estimation in large portfolios," Papers 1912.05484, arXiv.org, revised Apr 2022.
    4. Chao Zheng & Jiangtao Pan, 2023. "Unbiased estimators for the Heston model with stochastic interest rates," Papers 2301.12072, arXiv.org, revised Aug 2023.
    5. Shuaiqiang Liu & Lech A. Grzelak & Cornelis W. Oosterlee, 2022. "The Seven-League Scheme: Deep Learning for Large Time Step Monte Carlo Simulations of Stochastic Differential Equations," Risks, MDPI, vol. 10(3), pages 1-27, February.
    6. Nabil Kahale, 2018. "General multilevel Monte Carlo methods for pricing discretely monitored Asian options," Papers 1805.09427, arXiv.org, revised Sep 2018.
    7. Nabil Kahalé, 2020. "Randomized Dimension Reduction for Monte Carlo Simulations," Management Science, INFORMS, vol. 66(3), pages 1421-1439, March.
    8. Cui, Zhenyu & Fu, Michael C. & Peng, Yijie & Zhu, Lingjiong, 2020. "Optimal unbiased estimation for expected cumulative discounted cost," European Journal of Operational Research, Elsevier, vol. 286(2), pages 604-618.
    9. Zhou, Zhengqing & Wang, Guanyang & Blanchet, Jose H. & Glynn, Peter W., 2023. "Unbiased Optimal Stopping via the MUSE," Stochastic Processes and their Applications, Elsevier, vol. 166(C).
    10. Zhengqing Zhou & Guanyang Wang & Jose Blanchet & Peter W. Glynn, 2021. "Unbiased Optimal Stopping via the MUSE," Papers 2106.02263, arXiv.org, revised Dec 2022.
    11. Ruzayqat Hamza M. & Jasra Ajay, 2020. "Unbiased estimation of the solution to Zakai’s equation," Monte Carlo Methods and Applications, De Gruyter, vol. 26(2), pages 113-129, June.
    12. Devang Sinha & Siddhartha P. Chakrabarty, 2022. "Multilevel Richardson-Romberg and Importance Sampling in Derivative Pricing," Papers 2209.00821, arXiv.org.
    13. Beskos, Alexandros & Jasra, Ajay & Law, Kody & Tempone, Raul & Zhou, Yan, 2017. "Multilevel sequential Monte Carlo samplers," Stochastic Processes and their Applications, Elsevier, vol. 127(5), pages 1417-1440.
    14. Kahalé, Nabil, 2020. "General multilevel Monte Carlo methods for pricing discretely monitored Asian options," European Journal of Operational Research, Elsevier, vol. 287(2), pages 739-748.
    15. Matti Vihola & Jouni Helske & Jordan Franks, 2020. "Importance sampling type estimators based on approximate marginal Markov chain Monte Carlo," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1339-1376, December.
    16. Pierre E. Jacob & John O’Leary & Yves F. Atchadé, 2020. "Unbiased Markov chain Monte Carlo methods with couplings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(3), pages 543-600, July.
    17. Yasa Syed & Guanyang Wang, 2023. "Optimal randomized multilevel Monte Carlo for repeatedly nested expectations," Papers 2301.04095, arXiv.org, revised May 2023.
    18. Goda, Takashi & Kitade, Wataru, 2023. "Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 204(C), pages 743-763.
    19. Li, Chenxu & Wu, Linjia, 2019. "Exact simulation of the Ornstein–Uhlenbeck driven stochastic volatility model," European Journal of Operational Research, Elsevier, vol. 275(2), pages 768-779.
    20. Ajay Jasra & Kody Law & Carina Suciu, 2020. "Advanced Multilevel Monte Carlo Methods," International Statistical Review, International Statistical Institute, vol. 88(3), pages 548-579, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2209.14430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.