IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v68y2020i1p233-249.html
   My bibliography  Save this article

Faster Kriging: Facing High-Dimensional Simulators

Author

Listed:
  • Xuefei Lu

    (Department of Decision Sciences, Bocconi University, 20136 Milan, Italy)

  • Alessandro Rudi

    (The National Institute for Research in Computer Science and Automation (INRIA), École Normale Supérieure, Paris, France, 75012)

  • Emanuele Borgonovo

    (Bocconi Institute for Data Science and Analytics (BIDSA), 20136 Milan, Italy, Department of Decision Sciences, Bocconi University, 20136 Milan, Italy)

  • Lorenzo Rosasco

    (Department of Computer Science, Bioengineering, Robotics and Systems Engineering (DIBRIS), Università degli Studi di Genova, 16145 Genova, Italy, Laboratory for Computational and Statistical Learning (LCSL), Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, Istituto Italiano di Tecnologia, 16163 Genova, Italy)

Abstract

Kriging is one of the most widely used emulation methods in simulation. However, memory and time requirements potentially hinder its application to data sets generated by high-dimensional simulators. We borrow from the machine learning literature to propose a new algorithmic implementation of kriging that, while preserving prediction accuracy, notably reduces time and memory requirements. The theoretical and computational foundations of the algorithm are provided. The work then reports results of extensive numerical experiments to compare the performance of the proposed algorithm against current kriging implementations, on simulators of increasing dimensionality. Findings show notable savings in time and memory requirements that allow one to handle inputs across more that 10,000 dimensions.

Suggested Citation

  • Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
  • Handle: RePEc:inm:oropre:v:68:y:2020:i:1:p:233-249
    DOI: 10.1287/opre.2019.1860
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/opre.2019.1860
    Download Restriction: no

    File URL: https://libkey.io/10.1287/opre.2019.1860?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jalali, Hamed & Van Nieuwenhuyse, Inneke & Picheny, Victor, 2017. "Comparison of Kriging-based algorithms for simulation optimization with heterogeneous noise," European Journal of Operational Research, Elsevier, vol. 261(1), pages 279-301.
    2. Paul Glasserman & Zongjian Liu, 2010. "Sensitivity Estimates from Characteristic Functions," Operations Research, INFORMS, vol. 58(6), pages 1611-1623, December.
    3. Reuven Y. Rubinstein, 1989. "Sensitivity Analysis and Performance Extrapolation for Computer Simulation Models," Operations Research, INFORMS, vol. 37(1), pages 72-81, February.
    4. NESTEROV, Yurii, 2012. "Efficiency of coordinate descent methods on huge-scale optimization problems," LIDAM Reprints CORE 2511, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    5. Kleijnen, J.P.C. & van Beers, W.C.M., 2018. "Prediction for Big Data through Kriging : Small Sequential and One-Shot Designs," Other publications TiSEM b0504930-f518-44f7-908c-6, Tilburg University, School of Economics and Management.
    6. Jing Xie & Peter I. Frazier & Stephen E. Chick, 2016. "Bayesian Optimization via Simulation with Pairwise Sampling and Correlated Prior Beliefs," Operations Research, INFORMS, vol. 64(2), pages 542-559, April.
    7. Kleijnen, J.P.C. & van Beers, W.C.M., 2018. "Prediction for Big Data through Kriging : Small Sequential and One-Shot Designs," Discussion Paper 2018-022, Tilburg University, Center for Economic Research.
    8. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    9. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    10. Marrel, Amandine & Iooss, Bertrand & Laurent, Béatrice & Roustant, Olivier, 2009. "Calculations of Sobol indices for the Gaussian process metamodel," Reliability Engineering and System Safety, Elsevier, vol. 94(3), pages 742-751.
    11. R. W. Conway, 1963. "Some Tactical Problems in Digital Simulation," Management Science, INFORMS, vol. 10(1), pages 47-61, October.
    12. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    13. Roustant, Olivier & Ginsbourger, David & Deville, Yves, 2012. "DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i01).
    14. Harvey M. Wagner, 1995. "Global Sensitivity Analysis," Operations Research, INFORMS, vol. 43(6), pages 948-969, December.
    15. Erickson, Collin B. & Ankenman, Bruce E. & Sanchez, Susan M., 2018. "Comparison of Gaussian process modeling software," European Journal of Operational Research, Elsevier, vol. 266(1), pages 179-192.
    16. Gregory C. Critchfield & Keith E. Willard, 1986. "Probabilistic Analysis of Decision Trees Using Monte Carlo Simulation," Medical Decision Making, , vol. 6(2), pages 85-92, June.
    17. Manel Baucells & Emanuele Borgonovo, 2013. "Invariant Probabilistic Sensitivity Analysis," Management Science, INFORMS, vol. 59(11), pages 2536-2549, November.
    18. Imry Rosenbaum & Jeremy Staum, 2017. "Multilevel Monte Carlo Metamodeling," Operations Research, INFORMS, vol. 65(4), pages 1062-1077, August.
    19. Stephen E. Chick, 2001. "Input Distribution Selection for Simulation Experiments: Accounting for Input Uncertainty," Operations Research, INFORMS, vol. 49(5), pages 744-758, October.
    20. Bahar Biller & Barry L. Nelson, 2005. "Fitting Time-Series Input Processes for Simulation," Operations Research, INFORMS, vol. 53(3), pages 549-559, June.
    21. Kleijnen, Jack P. C. & van Beers, Wim C. M., 2005. "Robustness of Kriging when interpolating in random simulation with heterogeneous variances: Some experiments," European Journal of Operational Research, Elsevier, vol. 165(3), pages 826-834, September.
    22. Barry L. Nelson, 2004. "50th Anniversary Article: Stochastic Simulation Research in Management Science," Management Science, INFORMS, vol. 50(7), pages 855-868, July.
    23. R. W. Conway & B. M. Johnson & W. L. Maxwell, 1959. "Some Problems of Digital Systems Simulation," Management Science, INFORMS, vol. 6(1), pages 92-110, October.
    24. Kleijnen, J.P.C. & Bettonvil, B.W.M., 1997. "Searching for important factors in simulation models with many factors : Sequential bifurcation," Other publications TiSEM be826993-22f9-4cb3-89df-3, Tilburg University, School of Economics and Management.
    25. Saltelli A. & Tarantola S., 2002. "On the Relative Importance of Input Factors in Mathematical Models: Safety Assessment for Nuclear Waste Disposal," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 702-709, September.
    26. Richard E. Wendell, 2004. "Tolerance Sensitivity and Optimality Bounds in Linear Programming," Management Science, INFORMS, vol. 50(6), pages 797-803, June.
    27. Lihua Sun & L. Jeff Hong & Zhaolin Hu, 2014. "Balancing Exploitation and Exploration in Discrete Optimization via Simulation Through a Gaussian Process-Based Search," Operations Research, INFORMS, vol. 62(6), pages 1416-1438, December.
    28. Bahar Biller & Barry L. Nelson, 2008. "Evaluation of the ARTAFIT Method for Fitting Time-Series Input Processes for Simulation," INFORMS Journal on Computing, INFORMS, vol. 20(3), pages 485-498, August.
    29. Xi Chen & Kyoung-Kuk Kim, 2016. "Efficient VaR and CVaR Measurement via Stochastic Kriging," INFORMS Journal on Computing, INFORMS, vol. 28(4), pages 629-644, November.
    30. W C M van Beers & J P C Kleijnen, 2003. "Kriging for interpolation in random simulation," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(3), pages 255-262, March.
    31. Gramacy, Robert B., 2016. "laGP: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 72(i01).
    32. Bruce Ankenman & Barry L. Nelson & Jeremy Staum, 2010. "Stochastic Kriging for Simulation Metamodeling," Operations Research, INFORMS, vol. 58(2), pages 371-382, April.
    33. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    34. Vonk Noordegraaf, Antonie & Nielen, Mirjam & Kleijnen, Jack P. C., 2003. "Sensitivity analysis by experimental design and metamodelling: Case study on simulation in national animal disease control," European Journal of Operational Research, Elsevier, vol. 146(3), pages 433-443, May.
    35. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
    36. Durrande, N. & Ginsbourger, D. & Roustant, O. & Carraro, L., 2013. "ANOVA kernels and RKHS of zero mean functions for model-based sensitivity analysis," Journal of Multivariate Analysis, Elsevier, vol. 115(C), pages 57-67.
    37. Bahar Biller & Canan G. Corlu, 2011. "Accounting for Parameter Uncertainty in Large-Scale Stochastic Simulations with Correlated Inputs," Operations Research, INFORMS, vol. 59(3), pages 661-673, June.
    38. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    39. Wei Xie & Barry L. Nelson & Russell R. Barton, 2014. "A Bayesian Framework for Quantifying Uncertainty in Stochastic Simulation," Operations Research, INFORMS, vol. 62(6), pages 1439-1452, December.
    40. L. Jeff Hong & Barry L. Nelson & Jie Xu, 2015. "Discrete Optimization via Simulation," International Series in Operations Research & Management Science, in: Michael C Fu (ed.), Handbook of Simulation Optimization, edition 127, chapter 0, pages 9-44, Springer.
    41. Xi Chen & Bruce E. Ankenman & Barry L. Nelson, 2013. "Enhancing Stochastic Kriging Metamodels with Gradient Estimators," Operations Research, INFORMS, vol. 61(2), pages 512-528, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barde, Sylvain, 2024. "Bayesian estimation of large-scale simulation models with Gaussian process regression surrogates," Computational Statistics & Data Analysis, Elsevier, vol. 196(C).
    2. Zemin Zheng & Jie Zhang & Yang Li, 2022. "L 0 -Regularized Learning for High-Dimensional Additive Hazards Regression," INFORMS Journal on Computing, INFORMS, vol. 34(5), pages 2762-2775, September.
    3. Xing Liu & Enrico Zio & Emanuele Borgonovo & Elmar Plischke, 2024. "A Systematic Approach of Global Sensitivity Analysis and Its Application to a Model for the Quantification of Resilience of Interconnected Critical Infrastructures," Energies, MDPI, vol. 17(8), pages 1-24, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    2. Plischke, Elmar & Borgonovo, Emanuele, 2019. "Copula theory and probabilistic sensitivity analysis: Is there a connection?," European Journal of Operational Research, Elsevier, vol. 277(3), pages 1046-1059.
    3. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    4. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    5. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    6. Peter Salemi & Jeremy Staum & Barry L. Nelson, 2019. "Generalized Integrated Brownian Fields for Simulation Metamodeling," Operations Research, INFORMS, vol. 67(3), pages 874-891, May.
    7. Jalali, Hamed & Van Nieuwenhuyse, Inneke & Picheny, Victor, 2017. "Comparison of Kriging-based algorithms for simulation optimization with heterogeneous noise," European Journal of Operational Research, Elsevier, vol. 261(1), pages 279-301.
    8. Borgonovo, Emanuele & Hazen, Gordon B. & Jose, Victor Richmond R. & Plischke, Elmar, 2021. "Probabilistic sensitivity measures as information value," European Journal of Operational Research, Elsevier, vol. 289(2), pages 595-610.
    9. Elmar Plischke & Emanuele Borgonovo, 2020. "Fighting the Curse of Sparsity: Probabilistic Sensitivity Measures From Cumulative Distribution Functions," Risk Analysis, John Wiley & Sons, vol. 40(12), pages 2639-2660, December.
    10. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    11. Pesenti, Silvana M. & Millossovich, Pietro & Tsanakas, Andreas, 2019. "Reverse sensitivity testing: What does it take to break the model?," European Journal of Operational Research, Elsevier, vol. 274(2), pages 654-670.
    12. Katja Schilling & Daniel Bauer & Marcus C. Christiansen & Alexander Kling, 2020. "Decomposing Dynamic Risks into Risk Components," Management Science, INFORMS, vol. 66(12), pages 5738-5756, December.
    13. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    14. Heredia, María Belén & Prieur, Clémentine & Eckert, Nicolas, 2021. "Nonparametric estimation of aggregated Sobol’ indices: Application to a depth averaged snow avalanche model," Reliability Engineering and System Safety, Elsevier, vol. 212(C).
    15. Xin Yun & L. Jeff Hong & Guangxin Jiang & Shouyang Wang, 2019. "On gamma estimation via matrix kriging," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(5), pages 393-410, August.
    16. Zhou, Tianli & Fields, Evan & Osorio, Carolina, 2023. "A data-driven discrete simulation-based optimization algorithm for car-sharing service design," Transportation Research Part B: Methodological, Elsevier, vol. 178(C).
    17. Pedrielli, Giulia & Wang, Songhao & Ng, Szu Hui, 2020. "An extended Two-Stage Sequential Optimization approach: Properties and performance," European Journal of Operational Research, Elsevier, vol. 287(3), pages 929-945.
    18. Gordon Hazen & Emanuele Borgonovo & Xuefei Lu, 2023. "Information Density in Decision Analysis," Decision Analysis, INFORMS, vol. 20(2), pages 89-108, June.
    19. Helin Zhu & Tianyi Liu & Enlu Zhou, 2015. "Risk Quantification in Stochastic Simulation under Input Uncertainty," Papers 1507.06015, arXiv.org, revised Dec 2017.
    20. Isadora Antoniano‐Villalobos & Emanuele Borgonovo & Sumeda Siriwardena, 2018. "Which Parameters Are Important? Differential Importance Under Uncertainty," Risk Analysis, John Wiley & Sons, vol. 38(11), pages 2459-2477, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:68:y:2020:i:1:p:233-249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.