IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v47y2001i8p1133-1149.html
   My bibliography  Save this article

New Procedures to Select the Best Simulated System Using Common Random Numbers

Author

Listed:
  • Stephen E. Chick

    (Department of Industrial and Operations Engineering, The University of Michigan, 1205 Beal Avenue, Ann Arbor, Michigan 48109-2117)

  • Koichiro Inoue

    (Department of Industrial and Operations Engineering, The University of Michigan, 1205 Beal Avenue, Ann Arbor, Michigan 48109-2117)

Abstract

Although simulation is widely used to select the best of several alternative system designs, and common random numbers is an important tool for reducing the computation effort of simulation experiments, there are surprisingly few tools available to help a simulation practitioner select the best system when common random numbers are employed. This paper presents new two-stage procedures that use common random numbers to help identify the best simulated system. The procedures allow for screening and attempt to allocate additional replications to improve the value of information obtained during the second stage, rather than determining the number of replications required to provide a given probability of correct selection guarantee. The procedures allow decision makers to reduce either the expected opportunity cost associated with potentially selecting an inferior system, or the probability of incorrect selection. A small empirical study indicates that the new procedures outperform several procedures with respect to several criteria, and identifies potential areas for further improvement.

Suggested Citation

  • Stephen E. Chick & Koichiro Inoue, 2001. "New Procedures to Select the Best Simulated System Using Common Random Numbers," Management Science, INFORMS, vol. 47(8), pages 1133-1149, August.
    • Handle: RePEc:inm:ormnsc:v:47:y:2001:i:8:p:1133-1149
    DOI: 10.1287/mnsc.47.8.1133.10229
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.47.8.1133.10229
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.47.8.1133.10229?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kaufman, Gordon M, 1969. "Conditional Prediction and Unbiasedness in Structural Equations," Econometrica, Econometric Society, vol. 37(1), pages 44-49, January.
    2. Stephen E. Chick & Koichiro Inoue, 2001. "New Two-Stage and Sequential Procedures for Selecting the Best Simulated System," Operations Research, INFORMS, vol. 49(5), pages 732-743, October.
    3. Barry L. Nelson & Frank J. Matejcik, 1995. "Using Common Random Numbers for Indifference-Zone Selection and Multiple Comparisons in Simulation," Management Science, INFORMS, vol. 41(12), pages 1935-1945, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Other publications TiSEM a6910d11-f9bc-4246-b1a7-2, Tilburg University, School of Economics and Management.
    2. L. Jeff Hong, 2006. "Fully sequential indifference‐zone selection procedures with variance‐dependent sampling," Naval Research Logistics (NRL), John Wiley & Sons, vol. 53(5), pages 464-476, August.
    3. Diana M. Negoescu & Peter I. Frazier & Warren B. Powell, 2011. "The Knowledge-Gradient Algorithm for Sequencing Experiments in Drug Discovery," INFORMS Journal on Computing, INFORMS, vol. 23(3), pages 346-363, August.
    4. Peter Frazier & Warren Powell & Savas Dayanik, 2009. "The Knowledge-Gradient Policy for Correlated Normal Beliefs," INFORMS Journal on Computing, INFORMS, vol. 21(4), pages 599-613, November.
    5. Nakayama, Marvin K., 2007. "Fixed-width multiple-comparison procedures using common random numbers for steady-state simulations," European Journal of Operational Research, Elsevier, vol. 182(3), pages 1330-1349, November.
    6. Shing Chih Tsai & Jun Luo & Chi Ching Sung, 2017. "Combined variance reduction techniques in fully sequential selection procedures," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(6), pages 502-527, September.
    7. Michael C. Fu & Jian-Qiang Hu & Chun-Hung Chen & Xiaoping Xiong, 2007. "Simulation Allocation for Determining the Best Design in the Presence of Correlated Sampling," INFORMS Journal on Computing, INFORMS, vol. 19(1), pages 101-111, February.
    8. Yijie Peng & Chun-Hung Chen & Michael C. Fu & Jian-Qiang Hu & Ilya O. Ryzhov, 2021. "Efficient Sampling Allocation Procedures for Optimal Quantile Selection," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 230-245, January.
    9. Ilya O. Ryzhov & Warren B. Powell, 2011. "Information Collection on a Graph," Operations Research, INFORMS, vol. 59(1), pages 188-201, February.
    10. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    11. Jason R. W. Merrick, 2009. "Bayesian Simulation and Decision Analysis: An Expository Survey," Decision Analysis, INFORMS, vol. 6(4), pages 222-238, December.
    12. Chun-Hung Chen & Donghai He & Michael Fu & Loo Hay Lee, 2008. "Efficient Simulation Budget Allocation for Selecting an Optimal Subset," INFORMS Journal on Computing, INFORMS, vol. 20(4), pages 579-595, November.
    13. Lee, Loo Hay & Chew, Ek Peng & Teng, Suyan & Chen, Yankai, 2008. "Multi-objective simulation-based evolutionary algorithm for an aircraft spare parts allocation problem," European Journal of Operational Research, Elsevier, vol. 189(2), pages 476-491, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Justin Boesel & Barry L. Nelson & Seong-Hee Kim, 2003. "Using Ranking and Selection to “Clean Up” after Simulation Optimization," Operations Research, INFORMS, vol. 51(5), pages 814-825, October.
    2. Lee, Loo Hay & Chew, Ek Peng & Manikam, Puvaneswari, 2006. "A general framework on the simulation-based optimization under fixed computing budget," European Journal of Operational Research, Elsevier, vol. 174(3), pages 1828-1841, November.
    3. Michael C. Fu & Jian-Qiang Hu & Chun-Hung Chen & Xiaoping Xiong, 2007. "Simulation Allocation for Determining the Best Design in the Presence of Correlated Sampling," INFORMS Journal on Computing, INFORMS, vol. 19(1), pages 101-111, February.
    4. Sigrún Andradóttir & Andrei A. Prudius, 2009. "Balanced Explorative and Exploitative Search with Estimation for Simulation Optimization," INFORMS Journal on Computing, INFORMS, vol. 21(2), pages 193-208, May.
    5. Anton J. Kleywegt & Vijay S. Nori & Martin W. P. Savelsbergh, 2002. "The Stochastic Inventory Routing Problem with Direct Deliveries," Transportation Science, INFORMS, vol. 36(1), pages 94-118, February.
    6. Diana M. Negoescu & Peter I. Frazier & Warren B. Powell, 2011. "The Knowledge-Gradient Algorithm for Sequencing Experiments in Drug Discovery," INFORMS Journal on Computing, INFORMS, vol. 23(3), pages 346-363, August.
    7. Stephen E. Chick & Koichiro Inoue, 2001. "New Two-Stage and Sequential Procedures for Selecting the Best Simulated System," Operations Research, INFORMS, vol. 49(5), pages 732-743, October.
    8. Jie Xu & Barry L. Nelson & L. Jeff Hong, 2013. "An Adaptive Hyperbox Algorithm for High-Dimensional Discrete Optimization via Simulation Problems," INFORMS Journal on Computing, INFORMS, vol. 25(1), pages 133-146, February.
    9. Barry L. Nelson & David Goldsman, 2001. "Comparisons with a Standard in Simulation Experiments," Management Science, INFORMS, vol. 47(3), pages 449-463, March.
    10. Zhou, Tianli & Fields, Evan & Osorio, Carolina, 2023. "A data-driven discrete simulation-based optimization algorithm for car-sharing service design," Transportation Research Part B: Methodological, Elsevier, vol. 178(C).
    11. Yijie Peng & Chun-Hung Chen & Michael C. Fu & Jian-Qiang Hu, 2016. "Dynamic Sampling Allocation and Design Selection," INFORMS Journal on Computing, INFORMS, vol. 28(2), pages 195-208, May.
    12. Rafiei, Rezvan & Nourelfath, Mustapha & Gaudreault, Jonathan & De Santa-Eulalia, Luis Antonio & Bouchard, Mathieu, 2015. "Dynamic safety stock in co-production demand-driven wood remanufacturing mills: A case study," International Journal of Production Economics, Elsevier, vol. 165(C), pages 90-99.
    13. Stephen E. Chick & Yaozhong Wu, 2005. "Selection Procedures with Frequentist Expected Opportunity Cost Bounds," Operations Research, INFORMS, vol. 53(5), pages 867-878, October.
    14. Zhongshun Shi & Siyang Gao & Hui Xiao & Weiwei Chen, 2019. "A worst‐case formulation for constrained ranking and selection with input uncertainty," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(8), pages 648-662, December.
    15. Demet Batur & Lina Wang & F. Fred Choobineh, 2018. "Methods for System Selection Based on Sequential Mean–Variance Analysis," INFORMS Journal on Computing, INFORMS, vol. 30(4), pages 724-738, November.
    16. Daniel Russo, 2020. "Simple Bayesian Algorithms for Best-Arm Identification," Operations Research, INFORMS, vol. 68(6), pages 1625-1647, November.
    17. Jing Xie & Peter I. Frazier & Stephen E. Chick, 2016. "Bayesian Optimization via Simulation with Pairwise Sampling and Correlated Prior Beliefs," Operations Research, INFORMS, vol. 64(2), pages 542-559, April.
    18. Peter I. Frazier & Warren B. Powell, 2010. "Paradoxes in Learning and the Marginal Value of Information," Decision Analysis, INFORMS, vol. 7(4), pages 378-403, December.
    19. Zhongshun Shi & Yijie Peng & Leyuan Shi & Chun-Hung Chen & Michael C. Fu, 2022. "Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 557-568, January.
    20. Seong-Hee Kim & Barry L. Nelson, 2006. "On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation," Operations Research, INFORMS, vol. 54(3), pages 475-488, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:47:y:2001:i:8:p:1133-1149. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.