IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v307y2023i1p266-278.html
   My bibliography  Save this article

On the finite-sample statistical validity of adaptive fully sequential procedures

Author

Listed:
  • Cheng, Zhenxia
  • Luo, Jun
  • Wu, Ruijing

Abstract

We consider the simulation optimization problem of selecting the best system design from a finite set of alternatives, which is known as ranking and selection (R&S). Many fully sequential procedures have been proposed to solve the R&S problem using a static sampling rule in order to ensure a finite-sample statistical guarantee. In this paper, we develop fully sequential procedures that can incorporate various adaptive sampling rules, based on a modification of Paulson’s bound Paulson (1964), while still preserving the finite-sample guarantee. In particular, we propose an adaptive sampling rule that utilizes the consecutively updated sample mean and sample variance information by solving a minimization problem of the approximated total sample size. Finally, we demonstrate the efficiency of the proposed procedures with several existing procedures through extensive simulation experiments, and apply them to solve an ambulance dispatching problem.

Suggested Citation

  • Cheng, Zhenxia & Luo, Jun & Wu, Ruijing, 2023. "On the finite-sample statistical validity of adaptive fully sequential procedures," European Journal of Operational Research, Elsevier, vol. 307(1), pages 266-278.
    • Handle: RePEc:eee:ejores:v:307:y:2023:i:1:p:266-278
    DOI: 10.1016/j.ejor.2022.11.038
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221722008955
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2022.11.038?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Barry L. Nelson & Julie Swann & David Goldsman & Wheyming Song, 2001. "Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large," Operations Research, INFORMS, vol. 49(6), pages 950-963, December.
    2. Giulia Pedrielli & K. Selcuk Candan & Xilun Chen & Logan Mathesen & Alireza Inanalouganji & Jie Xu & Chun-Hung Chen & Loo Hay Lee, 2019. "Generalized Ordinal Learning Framework (GOLF) for Decision Making with Future Simulated Data," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 36(06), pages 1-35, December.
    3. Susan R. Hunter & Benjamin McClosky, 2016. "Maximizing quantitative traits in the mating design problem via simulation-based Pareto estimation," IISE Transactions, Taylor & Francis Journals, vol. 48(6), pages 565-578, June.
    4. Batur, D. & Choobineh, F., 2010. "A quantile-based approach to system selection," European Journal of Operational Research, Elsevier, vol. 202(3), pages 764-772, May.
    5. Weiwei Fan & L. Jeff Hong & Barry L. Nelson, 2016. "Indifference-Zone-Free Selection of the Best," Operations Research, INFORMS, vol. 64(6), pages 1499-1514, December.
    6. Tsai, Shing Chih & Chen, Sin Ting, 2017. "A simulation-based multi-objective optimization framework: A case study on inventory management," Omega, Elsevier, vol. 70(C), pages 148-159.
    7. Yijie Peng & Chun-Hung Chen & Michael C. Fu & Jian-Qiang Hu & Ilya O. Ryzhov, 2021. "Efficient Sampling Allocation Procedures for Optimal Quantile Selection," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 230-245, January.
    8. Lee, Mi Lim & Park, Chuljin & Park, Dong Uk, 2018. "Self-adjusting the tolerance level in a fully sequential feasibility check procedure," European Journal of Operational Research, Elsevier, vol. 271(2), pages 733-745.
    9. Daniel Russo, 2020. "Simple Bayesian Algorithms for Best-Arm Identification," Operations Research, INFORMS, vol. 68(6), pages 1625-1647, November.
    10. L. Jeff Hong, 2006. "Fully sequential indifference‐zone selection procedures with variance‐dependent sampling," Naval Research Logistics (NRL), John Wiley & Sons, vol. 53(5), pages 464-476, August.
    11. Ruijing Wu & Shaoxuan Liu & Zhenyang Shi, 2020. "Algorithm for Calculating the Initial Sample Size in a Fully Sequential Ranking and Selection Procedure," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 37(03), pages 1-19, May.
    12. Stephen E. Chick & Koichiro Inoue, 2001. "New Two-Stage and Sequential Procedures for Selecting the Best Simulated System," Operations Research, INFORMS, vol. 49(5), pages 732-743, October.
    13. Siyang Gao & Weiwei Chen & Leyuan Shi, 2017. "A New Budget Allocation Framework for the Expected Opportunity Cost," Operations Research, INFORMS, vol. 65(3), pages 787-803, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhongshun Shi & Yijie Peng & Leyuan Shi & Chun-Hung Chen & Michael C. Fu, 2022. "Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 557-568, January.
    2. Ye Chen & Ilya O. Ryzhov, 2023. "Balancing Optimal Large Deviations in Sequential Selection," Management Science, INFORMS, vol. 69(6), pages 3457-3473, June.
    3. Zhongshun Shi & Siyang Gao & Hui Xiao & Weiwei Chen, 2019. "A worst‐case formulation for constrained ranking and selection with input uncertainty," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(8), pages 648-662, December.
    4. Wang, Tianxiang & Xu, Jie & Hu, Jian-Qiang & Chen, Chun-Hung, 2023. "Efficient estimation of a risk measure requiring two-stage simulation optimization," European Journal of Operational Research, Elsevier, vol. 305(3), pages 1355-1365.
    5. Haihui Shen & L. Jeff Hong & Xiaowei Zhang, 2021. "Ranking and Selection with Covariates for Personalized Decision Making," INFORMS Journal on Computing, INFORMS, vol. 33(4), pages 1500-1519, October.
    6. Demet Batur & F. Fred Choobineh, 2021. "Selecting the Best Alternative Based on Its Quantile," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 657-671, May.
    7. Peter I. Frazier, 2014. "A Fully Sequential Elimination Procedure for Indifference-Zone Ranking and Selection with Tight Bounds on Probability of Correct Selection," Operations Research, INFORMS, vol. 62(4), pages 926-942, August.
    8. Dongwook Shin & Mark Broadie & Assaf Zeevi, 2022. "Practical Nonparametric Sampling Strategies for Quantile-Based Ordinal Optimization," INFORMS Journal on Computing, INFORMS, vol. 34(2), pages 752-768, March.
    9. Jie Xu & Barry L. Nelson & L. Jeff Hong, 2013. "An Adaptive Hyperbox Algorithm for High-Dimensional Discrete Optimization via Simulation Problems," INFORMS Journal on Computing, INFORMS, vol. 25(1), pages 133-146, February.
    10. Daniel Russo, 2020. "Simple Bayesian Algorithms for Best-Arm Identification," Operations Research, INFORMS, vol. 68(6), pages 1625-1647, November.
    11. Weiwei Fan & L. Jeff Hong & Barry L. Nelson, 2016. "Indifference-Zone-Free Selection of the Best," Operations Research, INFORMS, vol. 64(6), pages 1499-1514, December.
    12. Gongbo Zhang & Yijie Peng & Jianghua Zhang & Enlu Zhou, 2023. "Asymptotically Optimal Sampling Policy for Selecting Top- m Alternatives," INFORMS Journal on Computing, INFORMS, vol. 35(6), pages 1261-1285, November.
    13. Ying Zhong & Shaoxuan Liu & Jun Luo & L. Jeff Hong, 2022. "Speeding Up Paulson’s Procedure for Large-Scale Problems Using Parallel Computing," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 586-606, January.
    14. Shing Chih Tsai & Wu Hung Lin & Chia Cheng Wu & Shao Jen Weng & Ching Fen Tang, 2022. "Decision support algorithms for optimizing surgery start times considering the performance variation," Health Care Management Science, Springer, vol. 25(2), pages 208-221, June.
    15. Tsai, Shing Chih & Chu, I-Hao, 2012. "Controlled multistage selection procedures for comparison with a standard," European Journal of Operational Research, Elsevier, vol. 223(3), pages 709-721.
    16. Chen, E. Jack & Kelton, W. David, 2005. "Sequential selection procedures: Using sample means to improve efficiency," European Journal of Operational Research, Elsevier, vol. 166(1), pages 133-153, October.
    17. Kabirian, Alireza & Ólafsson, Sigurdur, 2011. "Continuous optimization via simulation using Golden Region search," European Journal of Operational Research, Elsevier, vol. 208(1), pages 19-27, January.
    18. L. Jeff Hong & Guangxin Jiang & Ying Zhong, 2022. "Solving Large-Scale Fixed-Budget Ranking and Selection Problems," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 2930-2949, November.
    19. Eric C. Ni & Dragos F. Ciocan & Shane G. Henderson & Susan R. Hunter, 2017. "Efficient Ranking and Selection in Parallel Computing Environments," Operations Research, INFORMS, vol. 65(3), pages 821-836, June.
    20. Michael C. Fu, 2002. "Simulation Optimization in the Future: Evolution or Revolution?," INFORMS Journal on Computing, INFORMS, vol. 14(3), pages 226-227, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:307:y:2023:i:1:p:266-278. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.