The use of simulation models in model driven experimentation

In model‐driven or model‐based experimentation, the model of the experiment is a key component of the closed loop model of the process. The model is created through interaction with the team designing the experimental organizations as well as the team creating the experimental environment. Starting with preliminary descriptions, the model evolves as more specific details are available and influences the final experimental design. The methodology used to design the model reflects both the types of design information available and the underlying hypothesis of the experiment. Experiments validating fixed types of structures or processes lead to a model designed with a structured analysis design technique which leads to an explicit but rigid model design. Experiments investigating adaptation require a more flexible model which can be created using an object oriented design approach. This leads to a more flexible, object view of the experimental design. Either approach leads to an appropriate set of models from which an executable model can be derived. The executable model is used to carry out simulations. In order to analyze the dynamic behavior of the model, an input scenario must be created based on the actual inputs that will be used in the experimental setting. When the model is stimulated with the scenario, its behavior can be observed and its performance measured on different criteria. Because it is a computer simulation, input parameters can be varied, constraints can be relaxed, and other variables (possibly) affecting the hypotheses can be explored to see their effect on the model and by inference the experiment. These results can then be made available to the design teams to influence further iterations of the design. Indeed, the model allows the consideration of many excursions, a situation that is not possible when the experiments include teams of humans. After the experiment is conducted, model validation is carried out by comparing the model results to the actual experimental results. This is done by driving the model with the original scenario, but including the actual decisions made by the human subjects, or decision makers. © 1999 John Wiley & Sons, Inc. Syst Eng 2: 108–128, 1999