IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v35y2023i2p495-508.html
   My bibliography  Save this article

SimOpt: A Testbed for Simulation-Optimization Experiments

Author

Listed:
  • David J. Eckman

    (Wm Michael Barnes ’64 Department of Industrial and Systems Engineering, Texas A&M University, College Station, Texas 77843)

  • Shane G. Henderson

    (School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853)

  • Sara Shashaani

    (Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, North Carolina 27695)

Abstract

This paper introduces a major redesign of SimOpt, a testbed of simulation-optimization (SO) problems and solvers. The testbed promotes the empirical evaluation and comparison of solvers and aims to accelerate their development. Relative to previous versions of SimOpt, the redesign ports the code to an object-oriented architecture in Python; uses an implementation of the MRG32k3a random number generator that supports streams, substreams, and subsubstreams; supports the automated use of common random numbers for ease and efficiency; includes a powerful suite of plotting tools for visualizing experiment results; uses bootstrapping to obtain error estimates; accommodates the use of data farming to explore simulation models and optimization solvers as their input parameters vary; and provides a graphical user interface. The SimOpt source code is available on a GitHub repository under a permissive open-source license and as a Python package.

Suggested Citation

  • David J. Eckman & Shane G. Henderson & Sara Shashaani, 2023. "SimOpt: A Testbed for Simulation-Optimization Experiments," INFORMS Journal on Computing, INFORMS, vol. 35(2), pages 495-508, March.
    • Handle: RePEc:inm:orijoc:v:35:y:2023:i:2:p:495-508
    DOI: 10.1287/ijoc.2023.1273
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2023.1273
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2023.1273?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Pierre L'Ecuyer & Richard Simard & E. Jack Chen & W. David Kelton, 2002. "An Object-Oriented Random-Number Package with Many Long Streams and Substreams," Operations Research, INFORMS, vol. 50(6), pages 1073-1075, December.
    2. David J. Eckman & Shane G. Henderson & Sara Shashaani, 2023. "Diagnostic Tools for Evaluating and Comparing Simulation-Optimization Algorithms," INFORMS Journal on Computing, INFORMS, vol. 35(2), pages 350-367, March.
    3. Kyle Cooper & Susan R. Hunter, 2020. "PyMOSO: Software for Multiobjective Simulation Optimization with R-PERLE and R-MinRLE," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 1101-1108, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pierre L'Ecuyer & Richard Simard, 2014. "On the Lattice Structure of a Special Class of Multiple Recursive Random Number Generators," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 449-460, August.
    2. Mehmet Tolga Cezik & Pierre L'Ecuyer, 2008. "Staffing Multiskill Call Centers via Linear Programming and Simulation," Management Science, INFORMS, vol. 54(2), pages 310-323, February.
    3. Iago Medeiros & Lucas Pacheco & Denis Rosário & Cristiano Both & Jéferson Nobre & Eduardo Cerqueira & Lisandro Granville, 2021. "Quality of experience and quality of service‐aware handover for video transmission in heterogeneous networks," International Journal of Network Management, John Wiley & Sons, vol. 31(5), September.
    4. Kampf, M. & Kochel, P., 2006. "Simulation-based sequencing and lot size optimisation for a production-and-inventory system with multiple items," International Journal of Production Economics, Elsevier, vol. 104(1), pages 191-200, November.
    5. Hiroshi Haramoto & Makoto Matsumoto & Takuji Nishimura & François Panneton & Pierre L'Ecuyer, 2008. "Efficient Jump Ahead for (F-openface) 2 -Linear Random Number Generators," INFORMS Journal on Computing, INFORMS, vol. 20(3), pages 385-390, August.
    6. Tang, Hui-Chin, 2005. "Reverse multiple recursive random number generators," European Journal of Operational Research, Elsevier, vol. 164(2), pages 402-405, July.
    7. Arthur Hau, 2011. "Pricing of Loan Commitments for Facilitating Stochastic Liquidity Needs," Journal of Financial Services Research, Springer;Western Finance Association, vol. 39(1), pages 71-94, April.
    8. Mascagni Michael & Hin Lin-Yee, 2013. "Parallel pseudo-random number generators: A derivative pricing perspective with the Heston stochastic volatility model," Monte Carlo Methods and Applications, De Gruyter, vol. 19(2), pages 77-105, July.
    9. Natasha Stout & Sue Goldie, 2008. "Keeping the noise down: common random numbers for disease simulation modeling," Health Care Management Science, Springer, vol. 11(4), pages 399-406, December.
    10. Céline Labart & Jérôme Lelong, 2011. "A Parallel Algorithm for solving BSDEs - Application to the pricing and hedging of American options," Working Papers hal-00567729, HAL.
    11. Imry Rosenbaum & Jeremy Staum, 2017. "Multilevel Monte Carlo Metamodeling," Operations Research, INFORMS, vol. 65(4), pages 1062-1077, August.
    12. Bivand, Roger, 2010. "Exploiting Parallelization in Spatial Statistics: an Applied Survey using R," Discussion Paper Series in Economics 25/2010, Norwegian School of Economics, Department of Economics.
    13. Gilles Pag`es & Benedikt Wilbertz, 2011. "GPGPUs in computational finance: Massive parallel computing for American style options," Papers 1101.3228, arXiv.org.
    14. Radu Herbei & L. Mark Berliner, 2014. "Estimating Ocean Circulation: An MCMC Approach With Approximated Likelihoods via the Bernoulli Factory," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 944-954, September.
    15. Eric C. Ni & Dragos F. Ciocan & Shane G. Henderson & Susan R. Hunter, 2017. "Efficient Ranking and Selection in Parallel Computing Environments," Operations Research, INFORMS, vol. 65(3), pages 821-836, June.
    16. Avramidis, Athanassios N. & Chan, Wyean & Gendreau, Michel & L'Ecuyer, Pierre & Pisacane, Ornella, 2010. "Optimizing daily agent scheduling in a multiskill call center," European Journal of Operational Research, Elsevier, vol. 200(3), pages 822-832, February.
    17. Kyle Cooper & Susan R. Hunter, 2020. "PyMOSO: Software for Multiobjective Simulation Optimization with R-PERLE and R-MinRLE," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 1101-1108, October.
    18. Thomas W. Lucas & W. David Kelton & Paul J. Sánchez & Susan M. Sanchez & Ben L. Anderson, 2015. "Changing the paradigm: Simulation, now a method of first resort," Naval Research Logistics (NRL), John Wiley & Sons, vol. 62(4), pages 293-303, June.
    19. Kastner, Gregor & Frühwirth-Schnatter, Sylvia, 2014. "Ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC estimation of stochastic volatility models," Computational Statistics & Data Analysis, Elsevier, vol. 76(C), pages 408-423.
    20. Kyle Cooper & Susan R. Hunter & Kalyani Nagaraj, 2020. "Biobjective Simulation Optimization on Integer Lattices Using the Epsilon-Constraint Method in a Retrospective Approximation Framework," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 1080-1100, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:35:y:2023:i:2:p:495-508. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.