IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v30y2018i4p724-738.html
   My bibliography  Save this article

Methods for System Selection Based on Sequential Mean–Variance Analysis

Author

Listed:
  • Demet Batur

    (Supply Chain Management and Analytics, University of Nebraska–Lincoln, Lincoln, Nebraska 68588)

  • Lina Wang

    (Supply Chain Management and Analytics, University of Nebraska–Lincoln, Lincoln, Nebraska 68588)

  • F. Fred Choobineh

    (Electrical and Computer Engineering, University of Nebraska–Lincoln, Lincoln, Nebraska 68588)

Abstract

We propose two sequential, indifference-zone procedures for the comparison of simulated systems. Comparisons and selection of the best system are based on the mean and variance of a performance metric estimated by simulation. The mean represents the central tendency while the variance is the surrogate for the system’s inherent systematic risk. The first procedure identifies the system(s) with the largest expected value and smallest variance. The second procedure uses the variance of a reference system as a risk threshold, and selects the system with the largest mean from among those with an acceptable level of risk not above the threshold. Numerical experiments demonstrate the validity and efficacy of the proposed procedures. The online appendix is available at https://doi.org/10.1287/ijoc.2018.0808 .

Suggested Citation

  • Demet Batur & Lina Wang & F. Fred Choobineh, 2018. "Methods for System Selection Based on Sequential Mean–Variance Analysis," INFORMS Journal on Computing, INFORMS, vol. 30(4), pages 724-738, November.
    • Handle: RePEc:inm:orijoc:v:30:y:2018:i:4:p:724-738
    DOI: 10.1287/ijoc.2018.0808
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2018.0808
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2018.0808?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Butler & Douglas J. Morrice & Peter W. Mullarkey, 2001. "A Multiple Attribute Utility Theory Approach to Ranking and Selection," Management Science, INFORMS, vol. 47(6), pages 800-816, June.
    2. Barry L. Nelson & David Goldsman, 2001. "Comparisons with a Standard in Simulation Experiments," Management Science, INFORMS, vol. 47(3), pages 449-463, March.
    3. Sigrún Andradóttir & Seong‐Hee Kim, 2010. "Fully sequential procedures for comparing constrained systems via simulation," Naval Research Logistics (NRL), John Wiley & Sons, vol. 57(5), pages 403-421, August.
    4. Loo Lee & Ek Chew & Suyan Teng & David Goldsman, 2010. "Finding the non-dominated Pareto set for multi-objective simulation models," IISE Transactions, Taylor & Francis Journals, vol. 42(9), pages 656-674.
    5. Seong-Hee Kim & Barry L. Nelson, 2006. "On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation," Operations Research, INFORMS, vol. 54(3), pages 475-488, June.
    6. David Goldsman & Seong-Hee Kim & William S. Marshall & Barry L. Nelson, 2002. "Ranking and Selection for Steady-State Simulation: Procedures and Perspectives," INFORMS Journal on Computing, INFORMS, vol. 14(1), pages 2-19, February.
    7. Susan R. Hunter & Raghu Pasupathy, 2013. "Optimal Sampling Laws for Stochastically Constrained Simulation Optimization on Finite Sets," INFORMS Journal on Computing, INFORMS, vol. 25(3), pages 527-542, August.
    8. Stephen E. Chick & Koichiro Inoue, 2001. "New Two-Stage and Sequential Procedures for Selecting the Best Simulated System," Operations Research, INFORMS, vol. 49(5), pages 732-743, October.
    9. Batur, D. & Choobineh, F., 2012. "Stochastic dominance based comparison for system selection," European Journal of Operational Research, Elsevier, vol. 220(3), pages 661-672.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Demet Batur & F. Fred Choobineh, 2021. "Selecting the Best Alternative Based on Its Quantile," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 657-671, May.
    2. Shing Chih Tsai & Wu Hung Lin & Chia Cheng Wu & Shao Jen Weng & Ching Fen Tang, 2022. "Decision support algorithms for optimizing surgery start times considering the performance variation," Health Care Management Science, Springer, vol. 25(2), pages 208-221, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. L. Jeff Hong & Jun Luo & Barry L. Nelson, 2015. "Chance Constrained Selection of the Best," INFORMS Journal on Computing, INFORMS, vol. 27(2), pages 317-334, May.
    2. Juergen Branke & Wen Zhang, 2019. "Identifying efficient solutions via simulation: myopic multi-objective budget allocation for the bi-objective case," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 41(3), pages 831-865, September.
    3. Zhongshun Shi & Yijie Peng & Leyuan Shi & Chun-Hung Chen & Michael C. Fu, 2022. "Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 557-568, January.
    4. Demet Batur & F. Fred Choobineh, 2021. "Selecting the Best Alternative Based on Its Quantile," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 657-671, May.
    5. Jürgen Branke & Stephen E. Chick & Christian Schmidt, 2007. "Selecting a Selection Procedure," Management Science, INFORMS, vol. 53(12), pages 1916-1932, December.
    6. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    7. Mattila, V. & Virtanen, K., 2015. "Ranking and selection for multiple performance measures using incomplete preference information," European Journal of Operational Research, Elsevier, vol. 242(2), pages 568-579.
    8. Zhongshun Shi & Siyang Gao & Hui Xiao & Weiwei Chen, 2019. "A worst‐case formulation for constrained ranking and selection with input uncertainty," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(8), pages 648-662, December.
    9. Healey, Christopher M. & Andradóttir, Sigrún & Kim, Seong-Hee, 2013. "Efficient comparison of constrained systems using dormancy," European Journal of Operational Research, Elsevier, vol. 224(2), pages 340-352.
    10. Peter I. Frazier, 2014. "A Fully Sequential Elimination Procedure for Indifference-Zone Ranking and Selection with Tight Bounds on Probability of Correct Selection," Operations Research, INFORMS, vol. 62(4), pages 926-942, August.
    11. Batur, D. & Choobineh, F., 2010. "A quantile-based approach to system selection," European Journal of Operational Research, Elsevier, vol. 202(3), pages 764-772, May.
    12. Lee, Loo Hay & Chew, Ek Peng & Manikam, Puvaneswari, 2006. "A general framework on the simulation-based optimization under fixed computing budget," European Journal of Operational Research, Elsevier, vol. 174(3), pages 1828-1841, November.
    13. Flötteröd, Gunnar, 2017. "A search acceleration method for optimization problems with transport simulation constraints," Transportation Research Part B: Methodological, Elsevier, vol. 98(C), pages 239-260.
    14. Yuwei Zhou & Sigrún Andradóttir & Seong-Hee Kim & Chuljin Park, 2022. "Finding Feasible Systems for Subjective Constraints Using Recycled Observations," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3080-3095, November.
    15. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    16. Chuljin Park & Seong-Hee Kim, 2015. "Penalty Function with Memory for Discrete Optimization via Simulation with Stochastic Constraints," Operations Research, INFORMS, vol. 63(5), pages 1195-1212, October.
    17. Akdogan, M. Altan & Bayındır, Z. Pelin & Iyigun, Cem, 2023. "An analysis of ambulance location problem from an equity perspective," Socio-Economic Planning Sciences, Elsevier, vol. 90(C).
    18. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Discussion Paper 2003-1, Tilburg University, Center for Economic Research.
    19. Satyajith Amaran & Nikolaos V. Sahinidis & Bikram Sharda & Scott J. Bury, 2016. "Simulation optimization: a review of algorithms and applications," Annals of Operations Research, Springer, vol. 240(1), pages 351-380, May.
    20. Gongbo Zhang & Yijie Peng & Jianghua Zhang & Enlu Zhou, 2023. "Asymptotically Optimal Sampling Policy for Selecting Top- m Alternatives," INFORMS Journal on Computing, INFORMS, vol. 35(6), pages 1261-1285, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:30:y:2018:i:4:p:724-738. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.