IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v279y2019i3p869-881.html
   My bibliography  Save this article

Detecting bias due to input modelling in computer simulation

Author

Listed:
  • Morgan, Lucy E.
  • Nelson, Barry L.
  • Titman, Andrew C.
  • Worthington, David J.

Abstract

This is the first paper to approach the problem of bias in the output of a stochastic simulation due to using input distributions whose parameters were estimated from real-world data. We consider, in particular, the bias in simulation-based estimators of the expected value (long-run average) of the real-world system performance; this bias will be present even if one employs unbiased estimators of the input distribution parameters due to the (typically) nonlinear relationship between these parameters and the output response. To date this bias has been assumed to be negligible because it decreases rapidly as the quantity of real-world input data increases. While true asymptotically, this property does not imply that the bias is actually small when, as is always the case, data are finite. We present a delta-method approach to bias estimation that evaluates the nonlinearity of the expected-value performance surface as a function of the input-model parameters. Since this response surface is unknown, we propose an innovative experimental design to fit a response-surface model that facilitates a test for detecting a bias of a relevant size with specified power. We evaluate the method using controlled experiments, and demonstrate it through a realistic case study concerning a healthcare call centre.

Suggested Citation

  • Morgan, Lucy E. & Nelson, Barry L. & Titman, Andrew C. & Worthington, David J., 2019. "Detecting bias due to input modelling in computer simulation," European Journal of Operational Research, Elsevier, vol. 279(3), pages 869-881.
  • Handle: RePEc:eee:ejores:v:279:y:2019:i:3:p:869-881
    DOI: 10.1016/j.ejor.2019.06.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221719304783
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2019.06.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kleijnen, Jack P. C., 1983. "Cross-validation using the t statistic," European Journal of Operational Research, Elsevier, vol. 13(2), pages 133-141, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lucy E. Morgan & Luke Rhodes-Leader & Russell R. Barton, 2022. "Reducing and Calibrating for Input Model Bias in Computer Simulation," INFORMS Journal on Computing, INFORMS, vol. 34(4), pages 2368-2382, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kleijnen, J.P.C., 1982. "Regression metamodel summarization of model behaviour," Other publications TiSEM 5ff81a8b-a5ca-4405-8606-e, Tilburg University, School of Economics and Management.
    2. Jack P. C. Kleijnen & Wim C. M. van Beers, 2022. "Statistical Tests for Cross-Validation of Kriging Models," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 607-621, January.
    3. Kleijnen, J.P.C., 2006. "Regression Models and Experimental Designs : A Tutorial for Simulation Analaysts," Other publications TiSEM 7b8ecddb-f49e-4b80-865b-a, Tilburg University, School of Economics and Management.
    4. Kleijnen, J.P.C., 1983. "Regression analysis : Assumptions, alternatives, applications," Research Memorandum FEW 135, Tilburg University, School of Economics and Management.
    5. Theuns, H.L. & Passier-Grootjans, A.M.L., 1982. "Internationaal toerisme : Een gids in de algemene basisliteratuur en het bronnenmateriaal," Research Memorandum FEW 124, Tilburg University, School of Economics and Management.
    6. Kleijnen, J.P.C., 1985. "A scenario for sequential experimentation," Other publications TiSEM 66e3aece-fa85-4afd-817c-4, Tilburg University, School of Economics and Management.
    7. Kleijnen, J.P.C., 1990. "Regression metamodels for simulation with common random numbers : Comparison of techniques," Research Memorandum FEW 426, Tilburg University, School of Economics and Management.
    8. Kleijnen, J.P.C., 1988. "Analyzing simulation experiments with common random number, part II : Rao's approach," Other publications TiSEM 18f1a3e3-e82a-4c21-b3bf-b, Tilburg University, School of Economics and Management.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:279:y:2019:i:3:p:869-881. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.