IDEAS home Printed from https://ideas.repec.org/a/pal/jorsoc/v55y2004i8d10.1057_palgrave.jors.2601747.html
   My bibliography  Save this article

Application-driven sequential designs for simulation experiments: Kriging metamodelling

Author

Listed:
  • J P C Kleijnen

    (Center for Economic Research (CentER), Tilburg University (UvT))

  • W C M van Beers

    (Center for Economic Research (CentER), Tilburg University (UvT))

Abstract

This paper proposes a novel method to select an experimental design for interpolation in simulation. Although the paper focuses on Kriging in deterministic simulation, the method also applies to other types of metamodels (besides Kriging), and to stochastic simulation. The paper focuses on simulations that require much computer time, so it is important to select a design with a small number of observations. The proposed method is therefore sequential. The novelty of the method is that it accounts for the specific input/output function of the particular simulation model at hand; that is, the method is application-driven or customized. This customization is achieved through cross-validation and jackknifing. The new method is tested through two academic applications, which demonstrate that the method indeed gives better results than either sequential designs based on an approximate Kriging prediction variance formula or designs with prefixed sample sizes.

Suggested Citation

  • J P C Kleijnen & W C M van Beers, 2004. "Application-driven sequential designs for simulation experiments: Kriging metamodelling," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 55(8), pages 876-883, August.
  • Handle: RePEc:pal:jorsoc:v:55:y:2004:i:8:d:10.1057_palgrave.jors.2601747
    DOI: 10.1057/palgrave.jors.2601747
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/palgrave.jors.2601747
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1057/palgrave.jors.2601747?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. W C M van Beers & J P C Kleijnen, 2003. "Kriging for interpolation in random simulation," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(3), pages 255-262, March.
    2. B. J. A. Mertens, 2001. "Downdating: Interdisciplinary Research Between Statistics and Computing," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 55(3), pages 358-366, November.
    3. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    4. Sungmin Park & John W. Fowler & Gerald T. Mackulak & J. Bert Keats & W. Matthew Carlyle, 2002. "D-Optimal Sequential Experiments for Generating a Simulation-Based Cycle Time-Throughput Curve," Operations Research, INFORMS, vol. 50(6), pages 981-990, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kleijnen, J.P.C., 2007. "Simulation Experiments in Practice : Statistical Design and Regression Analysis," Discussion Paper 2007-09, Tilburg University, Center for Economic Research.
    2. Song, Kunling & Zhang, Yugang & Shen, Linjie & Zhao, Qingyan & Song, Bifeng, 2021. "A failure boundary exploration and exploitation framework combining adaptive Kriging model and sample space partitioning strategy for efficient reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    3. D den Hertog & J P C Kleijnen & A Y D Siem, 2006. "The correct Kriging variance estimated by bootstrapping," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 57(4), pages 400-409, April.
    4. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    5. Edwin Dam & Bart Husslage & Dick Hertog, 2010. "One-dimensional nested maximin designs," Journal of Global Optimization, Springer, vol. 46(2), pages 287-306, February.
    6. Kleijnen, J.P.C. & van Beers, W.C.M. & van Nieuwenhuyse, I., 2008. "Constrained Optimization in Simulation : A Novel Approach," Other publications TiSEM e49ba0fc-853c-4a13-b564-d, Tilburg University, School of Economics and Management.
    7. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    8. Kleijnen, J.P.C., 2006. "White Noise Assumptions Revisited : Regression Models and Statistical Designs for Simulation Practice," Other publications TiSEM d8c37ad3-f9a5-4824-986d-2, Tilburg University, School of Economics and Management.
    9. Xiao, Ning-Cong & Zuo, Ming J. & Zhou, Chengning, 2018. "A new adaptive sequential sampling method to construct surrogate models for efficient reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 330-338.
    10. Siem, A.Y.D. & den Hertog, D., 2007. "Kriging Models That Are Robust With Respect to Simulation Errors," Other publications TiSEM fe73dc8b-20d6-4f50-95eb-f, Tilburg University, School of Economics and Management.
    11. P. Pedone & G. Vicario & D. Romano, 2009. "Kriging‐based sequential inspection plans for coordinate measuring machines," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 25(2), pages 133-149, March.
    12. Kleijnen, J.P.C., 2007. "Screening Experiments for Simulation : A Review," Discussion Paper 2007-21, Tilburg University, Center for Economic Research.
    13. Jize Zhang & Alexandros A. Taflanidis & Norberto C. Nadal-Caraballo & Jeffrey A. Melby & Fatimata Diop, 2018. "Advances in surrogate modeling for storm surge prediction: storm selection and addressing characteristics related to climate change," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 94(3), pages 1225-1253, December.
    14. Maroussa Zagoraiou & Alessandro Baldi Antognini, 2009. "Optimal designs for parameter estimation of the Ornstein–Uhlenbeck process," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 25(5), pages 583-600, September.
    15. Gaofeng Jia & Alexandros A. Taflanidis & Norberto C. Nadal-Caraballo & Jeffrey A. Melby & Andrew B. Kennedy & Jane M. Smith, 2016. "Surrogate modeling for peak or time-dependent storm surge prediction over an extended coastal region using an existing database of synthetic storms," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 81(2), pages 909-938, March.
    16. Gan, Guojun & Lin, X. Sheldon, 2015. "Valuation of large variable annuity portfolios under nested simulation: A functional data approach," Insurance: Mathematics and Economics, Elsevier, vol. 62(C), pages 138-150.
    17. Mlakar, Miha & Petelin, Dejan & Tušar, Tea & Filipič, Bogdan, 2015. "GP-DEMO: Differential Evolution for Multiobjective Optimization based on Gaussian Process models," European Journal of Operational Research, Elsevier, vol. 243(2), pages 347-361.
    18. Stinstra, Erwin & den Hertog, Dick, 2008. "Robust optimization using computer experiments," European Journal of Operational Research, Elsevier, vol. 191(3), pages 816-837, December.
    19. Gaofeng Jia & Alexandros Taflanidis & Norberto Nadal-Caraballo & Jeffrey Melby & Andrew Kennedy & Jane Smith, 2016. "Surrogate modeling for peak or time-dependent storm surge prediction over an extended coastal region using an existing database of synthetic storms," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 81(2), pages 909-938, March.
    20. Stinstra, E., 2006. "The meta-model approach for simulation-based design optimization," Other publications TiSEM 713f828a-4716-4a19-af00-e, Tilburg University, School of Economics and Management.
    21. Xin, Fukang & Wang, Pan & Wang, Qirui & Li, Lei & Cheng, Lei & Lei, Huajin & Ma, Fangyun, 2024. "Parallel adaptive ensemble of metamodels combined with hypersphere sampling for rare failure events," Reliability Engineering and System Safety, Elsevier, vol. 246(C).
    22. M I Reis dos Santos & P M Reis dos Santos, 2011. "Construction and validation of distribution-based regression simulation metamodels," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(7), pages 1376-1384, July.
    23. Daniel J Klein & Michael Baym & Philip Eckhoff, 2014. "The Separatrix Algorithm for Synthesis and Analysis of Stochastic Simulations with Applications in Disease Modeling," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-10, July.
    24. Stinstra, E. & den Hertog, D., 2005. "Robust Optimization Using Computer Experiments," Other publications TiSEM 69d6e378-c9f9-44e8-9602-f, Tilburg University, School of Economics and Management.
    25. Scott L. Rosen & Christopher P. Saunders & Samar K Guharay, 2015. "A Structured Approach for Rapidly Mapping Multilevel System Measures via Simulation Metamodeling," Systems Engineering, John Wiley & Sons, vol. 18(1), pages 87-101, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    2. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    3. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    4. van Beers, Wim C.M. & Kleijnen, Jack P.C., 2008. "Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping," European Journal of Operational Research, Elsevier, vol. 186(3), pages 1099-1113, May.
    5. Feng Yang & Bruce Ankenman & Barry L. Nelson, 2007. "Efficient generation of cycle time‐throughput curves through simulation and metamodeling," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(1), pages 78-93, February.
    6. Kleijnen, Jack P.C. & Deflandre, David, 2006. "Validation of regression metamodels in simulation: Bootstrap approach," European Journal of Operational Research, Elsevier, vol. 170(1), pages 120-131, April.
    7. M I Reis dos Santos & P M Reis dos Santos, 2011. "Construction and validation of distribution-based regression simulation metamodels," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(7), pages 1376-1384, July.
    8. Katarzyna Growiec & Jakub Growiec & Bogumil Kaminski, 2017. "Social Network Structure and The Trade-Off Between Social Utility and Economic Performance," KAE Working Papers 2017-026, Warsaw School of Economics, Collegium of Economic Analysis.
    9. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    10. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    11. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    12. H. Christopher Frey & Sumeet R. Patil, 2002. "Identification and Review of Sensitivity Analysis Methods," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 553-578, June.
    13. Tunali, S. & Batmaz, I., 2003. "A metamodeling methodology involving both qualitative and quantitative input factors," European Journal of Operational Research, Elsevier, vol. 150(2), pages 437-450, October.
    14. Panknin, Lea & Boy, Karl-Friedrich & Henning, Christian H.C.A., 2024. "Can the European Green Deal be a game changer for sustainable food system transformation? A computational political economy approach," 2024 Annual Meeting, July 28-30, New Orleans, LA 343740, Agricultural and Applied Economics Association.
    15. Jin, Ding & Thube, Sneha Dattatraya & Hedtrich, Johannes & Henning, Christian & Delzeit, Ruth, 2019. "A Baseline Calibration Procedure for CGE models: An Application for DART," Conference papers 333057, Purdue University, Center for Global Trade Analysis, Global Trade Analysis Project.
    16. Robertson, Joseph J. & Polly, Ben J. & Collis, Jon M., 2015. "Reduced-order modeling and simulated annealing optimization for efficient residential building utility bill calibration," Applied Energy, Elsevier, vol. 148(C), pages 169-177.
    17. Richard E. Nance & Robert G. Sargent, 2002. "Perspectives on the Evolution of Simulation," Operations Research, INFORMS, vol. 50(1), pages 161-172, February.
    18. Durieux, Severine & Pierreval, Henri, 2004. "Regression metamodeling for the design of automated manufacturing system composed of parallel machines sharing a material handling resource," International Journal of Production Economics, Elsevier, vol. 89(1), pages 21-30, May.
    19. Nariman Fouladinejad & Nima Fouladinejad & Mohamad Kasim Abdul Jalil & Jamaludin Mohd Taib, 2017. "Decomposition-Assisted Computational Technique Based on Surrogate Modeling for Real-Time Simulations," Complexity, Hindawi, vol. 2017, pages 1-14, March.
    20. Ziesmer, Johannes & Jin, Ding & Mukashov, Askar & Henning, Christian, 2023. "Integrating fundamental model uncertainty in policy analysis," Socio-Economic Planning Sciences, Elsevier, vol. 87(PB).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:jorsoc:v:55:y:2004:i:8:d:10.1057_palgrave.jors.2601747. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.palgrave-journals.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.