IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v313y2024i1p241-257.html
   My bibliography  Save this article

Generating and validating cluster sampling matrices for model-free factor screening

Author

Listed:
  • Shi, Wen
  • Chen, Ao
  • Xie, Xiang

Abstract

Morris’ elementary effect-based screening (MM) has been widely used in a variety of domains to identify a few important factors among many possible ones. In MM, cluster sampling offers substantial computational savings over non-cluster sampling, but it remains a challenge to construct a cluster sampling matrix that generates any particular number of elementary effects for each factor. In this paper, we thoroughly address this issue. We uncover the mathematical association between distinct block sampling matrices within the complete cluster sampling matrix, by introducing a “dummy” sub-block matrix. By leveraging this property, we propose an easy-to-implement adaptive cluster sampling (ACS) method that is capable of identifying the appropriate sub-block sampling matrix to use. Its advantage over existing brute-force methods is that it can provide easy-to-obtain cluster sampling matrices, and it can be applied to computational model with any number of factors given a prior. We demonstrate the attractive properties of ACS using analytical proofs and simulation experiments. We show the robustness of ACS via a real-world case study. Our code for the algorithm is available online.

Suggested Citation

  • Shi, Wen & Chen, Ao & Xie, Xiang, 2024. "Generating and validating cluster sampling matrices for model-free factor screening," European Journal of Operational Research, Elsevier, vol. 313(1), pages 241-257.
    • Handle: RePEc:eee:ejores:v:313:y:2024:i:1:p:241-257
    DOI: 10.1016/j.ejor.2023.08.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221723006112
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2023.08.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    2. Pujol, Gilles, 2009. "Simplex-based screening designs for estimating metamodels," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1156-1160.
    3. James A. Bradley & Dominik Hülse & Douglas E. LaRowe & Sandra Arndt, 2022. "Transfer efficiency of organic carbon in marine sediments," Nature Communications, Nature, vol. 13(1), pages 1-8, December.
    4. Hua Shen & Hong Wan & Susan M. Sanchez, 2010. "A hybrid method for simulation factor screening," Naval Research Logistics (NRL), John Wiley & Sons, vol. 57(1), pages 45-57, February.
    5. Mavromatidis, Georgios & Orehounig, Kristina & Carmeliet, Jan, 2018. "Design of distributed energy systems under uncertainty: A two-stage stochastic programming approach," Applied Energy, Elsevier, vol. 222(C), pages 932-950.
    6. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    7. Borgonovo, Emanuele & Rabitti, Giovanni, 2023. "Screening: From tornado diagrams to effective dimensions," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1200-1211.
    8. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2006. "Controlled Sequential Bifurcation: A New Factor-Screening Method for Discrete-Event Simulation," Operations Research, INFORMS, vol. 54(4), pages 743-755, August.
    9. Emanuele Borgonovo & Marco Pangallo & Jan Rivkin & Leonardo Rizzo & Nicolaj Siggelkow, 2022. "Sensitivity analysis of agent-based models: a new protocol," Computational and Mathematical Organization Theory, Springer, vol. 28(1), pages 52-94, March.
    10. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    11. R G Sargent, 2013. "Verification and validation of simulation models," Journal of Simulation, Taylor & Francis Journals, vol. 7(1), pages 12-24, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    2. Shi, Wen & Zhou, Qing & Zhou, Yanju, 2023. "An efficient elementary effect-based method for sensitivity analysis in identifying main and two-factor interaction effects," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    3. Shi, Wen & Chen, Xi, 2019. "Controlled Morris method: A new factor screening approach empowered by a distribution-free sequential multiple testing procedure," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 299-314.
    4. Wen Shi & Xi Chen, 2018. "Efficient budget allocation strategies for elementary effects method in stochastic simulation," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(3), pages 218-241, April.
    5. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    6. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    7. Nicola Rossi & Mario Bačić & Lovorka Librić & Meho Saša Kovačević, 2023. "Methodology for Identification of the Key Levee Parameters for Limit-State Analyses Based on Sequential Bifurcation," Sustainability, MDPI, vol. 15(6), pages 1-16, March.
    8. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    9. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    10. Borgonovo, Emanuele & Clemente, Gian Paolo & Rabitti, Giovanni, 2024. "Why insurance regulators need to require sensitivity settings of internal models for their approval," Finance Research Letters, Elsevier, vol. 60(C).
    11. Becker, William, 2020. "Metafunctions for benchmarking in sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
    12. Martin Egozcue, 2024. "An Integer Optimization Approach for Determining Building Height," SN Operations Research Forum, Springer, vol. 5(3), pages 1-22, September.
    13. Makam, Vaishno Devi & Millossovich, Pietro & Tsanakas, Andreas, 2021. "Sensitivity analysis with χ2-divergences," Insurance: Mathematics and Economics, Elsevier, vol. 100(C), pages 372-383.
    14. Shen, Feifei & Zhao, Liang & Wang, Meihong & Du, Wenli & Qian, Feng, 2022. "Data-driven adaptive robust optimization for energy systems in ethylene plant under demand uncertainty," Applied Energy, Elsevier, vol. 307(C).
    15. F. Wang & G. H. Huang & Y. Fan & Y. P. Li, 2020. "Robust Subsampling ANOVA Methods for Sensitivity Analysis of Water Resource and Environmental Models," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 34(10), pages 3199-3217, August.
    16. Daniel Harenberg & Stefano Marelli & Bruno Sudret & Viktor Winschel, 2019. "Uncertainty quantification and global sensitivity analysis for economic models," Quantitative Economics, Econometric Society, vol. 10(1), pages 1-41, January.
    17. Magni, Carlo Alberto, 2016. "Capital depreciation and the underdetermination of rate of return: A unifying perspective," Journal of Mathematical Economics, Elsevier, vol. 67(C), pages 54-79.
    18. Lu, Xuefei & Borgonovo, Emanuele, 2023. "Global sensitivity analysis in epidemiological modeling," European Journal of Operational Research, Elsevier, vol. 304(1), pages 9-24.
    19. Matteo Fontana & Massimo Tavoni & Simone Vantini, 2020. "Global Sensitivity and Domain-Selective Testing for Functional-Valued Responses: An Application to Climate Economy Models," Papers 2006.13850, arXiv.org, revised Apr 2024.
    20. Jiang, Sheng-Long & Wang, Meihong & Bogle, I. David L., 2023. "Plant-wide byproduct gas distribution under uncertainty in iron and steel industry via quantile forecasting and robust optimization," Applied Energy, Elsevier, vol. 350(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:313:y:2024:i:1:p:241-257. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.