IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v44y1998i6p820-830.html
   My bibliography  Save this article

Active Nonlinear Tests (ANTs) of Complex Simulation Models

Author

Listed:
  • John H. Miller

    (Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213)

Abstract

Simulation models are becoming increasingly common in the analysis of critical scientific, policy, and management issues. Such models provide a way to analyze complex systems characterized by both large parameter spaces and nonlinear interactions. Unfortunately, these same characteristics make understanding such models using traditional testing techniques extremely difficult. Here we show how a model's structure and robustness can be validated via a simple, automatic, nonlinear search algorithm designed to actively "break" the model's implications. Using the active nonlinear tests (ANTs) developed here, one can easily probe for key weaknesses in a simulation's structure, and thereby begin to improve and refine its design. We demonstrate ANTs by testing a well-known model of global dynamics (World3), and show how this technique can be used to uncover small, but powerful, nonlinear effects that may highlight vulnerabilities in the original model.

Suggested Citation

  • John H. Miller, 1998. "Active Nonlinear Tests (ANTs) of Complex Simulation Models," Management Science, INFORMS, vol. 44(6), pages 820-830, June.
    • Handle: RePEc:inm:ormnsc:v:44:y:1998:i:6:p:820-830
    DOI: 10.1287/mnsc.44.6.820
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.44.6.820
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.44.6.820?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Holland, John H & Miller, John H, 1991. "Artificial Adaptive Agents in Economic Theory," American Economic Review, American Economic Association, vol. 81(2), pages 365-371, May.
    2. Steve Bankes, 1993. "Exploratory Modeling for Policy Analysis," Operations Research, INFORMS, vol. 41(3), pages 435-449, June.
    3. Nordhaus, William D, 1973. "World Dynamics: Measurement Without Data," Economic Journal, Royal Economic Society, vol. 83(332), pages 1156-1183, December.
    4. William D. Nordhaus, 1992. "Lethal Model 2: The Limits to Growth Revisited," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 23(2), pages 1-60.
    5. David C. Cox & Paul Baybutt, 1981. "Methods for Uncertainty Analysis: A Comparative Survey," Risk Analysis, John Wiley & Sons, vol. 1(4), pages 251-258, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Haruvy, Ernan & Roth, Alvin E. & Unver, M. Utku, 2006. "The dynamics of law clerk matching: An experimental and computational investigation of proposals for reform of the market," Journal of Economic Dynamics and Control, Elsevier, vol. 30(3), pages 457-486, March.
    2. Carrella, Ernesto & Saul, Steven & Marshall, Kristin & Burgess, Matthew G. & Cabral, Reniel B. & Bailey, Richard M. & Dorsett, Chris & Drexler, Michael & Madsen, Jens Koed & Merkl, Andreas, 2020. "Simple Adaptive Rules Describe Fishing Behaviour Better than Perfect Rationality in the US West Coast Groundfish Fishery," Ecological Economics, Elsevier, vol. 169(C).
    3. Gönenç Yücel & Yaman Barlas, 2011. "Automated parameter specification in dynamic feedback models based on behavior pattern features," System Dynamics Review, System Dynamics Society, vol. 27(2), pages 195-215, April.
    4. William Tracy, 2014. "Paradox Lost: The Evolution of Strategies in Selten’s Chain Store Game," Computational Economics, Springer;Society for Computational Economics, vol. 43(1), pages 83-103, January.
    5. Oliva, Rogelio, 2003. "Model calibration as a testing strategy for system dynamics models," European Journal of Operational Research, Elsevier, vol. 151(3), pages 552-568, December.
    6. Komeil Mahjori Karmozdi & Mohammad Reza Kohansal & Mohammad Ghorbani, 2020. "Sustainable economic rural development system pattern in Ghaemshahr: an application of the developed TOP-MARD core model," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 22(6), pages 5793-5817, August.
    7. Ma, Tieju & Nakamori, Yoshiteru, 2005. "Agent-based modeling on technological innovation as an evolutionary process," European Journal of Operational Research, Elsevier, vol. 166(3), pages 741-755, November.
    8. Margolis, David N. & Navarro, Lucas & Robalino, David A., 2012. "Unemployment Insurance, Job Search and Informal Employment," IZA Discussion Papers 6660, Institute of Labor Economics (IZA).
    9. Kwakkel, Jan H. & Pruyt, Erik, 2013. "Exploratory Modeling and Analysis, an approach for model-based foresight under deep uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 419-431.
    10. Eva Labro, 2015. "Using simulation methods in accounting research," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 26(2), pages 99-104, August.
    11. Benjamin L. Turner & Hector M. Menendez & Roger Gates & Luis O. Tedeschi & Alberto S. Atzori, 2016. "System Dynamics Modeling for Agricultural and Natural Resource Management Issues: Review of Some Past Cases and Forecasting Future Roles," Resources, MDPI, vol. 5(4), pages 1-24, November.
    12. Keiki Takadama & Takao Terano & Katsunori Shimohara, 2003. "Interpretation by Implementation for Understanding a Multiagent Organization," Computational and Mathematical Organization Theory, Springer, vol. 9(1), pages 19-35, May.
    13. Sibel Eker & Jill Slinger & Els Daalen & Gönenç Yücel, 2014. "Sensitivity analysis of graphical functions," System Dynamics Review, System Dynamics Society, vol. 30(3), pages 186-205, July.
    14. Prenkert, Frans & Følgesvold, Atle, 2014. "Relationship strength and network form: An agent-based simulation of interaction in a business network," Australasian marketing journal, Elsevier, vol. 22(1), pages 15-27.
    15. Scott F. Turner & Richard A. Bettis & Richard M. Burton, 2002. "Exploring Depth Versus Breadth in Knowledge Management Strategies," Computational and Mathematical Organization Theory, Springer, vol. 8(1), pages 49-73, May.
    16. Jan H. Kwakkel & Erik Pruyt, 2015. "Using System Dynamics for Grand Challenges: The ESDMA Approach," Systems Research and Behavioral Science, Wiley Blackwell, vol. 32(3), pages 358-375, May.
    17. Chaturvedi, Alok & Mehta, Shailendra & Dolk, Daniel & Ayer, Rick, 2005. "Agent-based simulation for computational experimentation: Developing an artificial labor market," European Journal of Operational Research, Elsevier, vol. 166(3), pages 694-716, November.
    18. Robalino, David A. & Voetberg, Albertus & Picazo, Oscar, 2002. "The macroeconomic impacts of AIDS in Kenya estimating optimal reduction targets for the HIV/AIDS incidence rate," Journal of Policy Modeling, Elsevier, vol. 24(2), pages 195-218, May.
    19. Busby, J.S., 2019. "The co-evolution of competition and parasitism in the resource-based view: A risk model of product counterfeiting," European Journal of Operational Research, Elsevier, vol. 276(1), pages 300-313.
    20. Azad M. Madni & Michael Sievers, 2018. "Model‐based systems engineering: Motivation, current status, and research opportunities," Systems Engineering, John Wiley & Sons, vol. 21(3), pages 172-190, May.
    21. Harald de Bruijn & Andreas Größler & Nuno Videira, 2020. "Antifragility as a design criterion for modelling dynamic systems," Systems Research and Behavioral Science, Wiley Blackwell, vol. 37(1), pages 23-37, January.
    22. Rand, William & Rust, Roland T., 2011. "Agent-based modeling in marketing: Guidelines for rigor," International Journal of Research in Marketing, Elsevier, vol. 28(3), pages 181-193.
    23. Erling Moxnes & Sergey Naumov, 2024. "Analysis of dynamic models by optimization," System Dynamics Review, System Dynamics Society, vol. 40(1), January.
    24. Peter Gerbrands & Brigitte Unger & Joras Ferwerda, 2022. "Bilateral responsive regulation and international tax competition: An agent‐based simulation," Regulation & Governance, John Wiley & Sons, vol. 16(3), pages 760-780, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. John H. Miller, 1996. "Active Nonlinear Tests (ANTs) of Complex Simulation Models," Working Papers 96-03-011, Santa Fe Institute.
    2. Robert S. Pindyck, 2017. "The Use and Misuse of Models for Climate Policy," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 11(1), pages 100-114.
    3. Cameron Hepburn & Alex Bowen, 2013. "Prosperity with growth: economic growth, climate change and environmental limits," Chapters, in: Roger Fouquet (ed.), Handbook on Energy and Climate Change, chapter 29, pages 617-638, Edward Elgar Publishing.
    4. Jeroen C.J.M. van den Bergh & Peter Nijkamp, 1997. "Optimal Growth, Coordination and Sustainability in the Spatial Economy," Tinbergen Institute Discussion Papers 97-104/3, Tinbergen Institute.
    5. van den Bergh, Jeroen C. J. M., 2004. "Optimal climate policy is a utopia: from quantitative to qualitative cost-benefit analysis," Ecological Economics, Elsevier, vol. 48(4), pages 385-393, April.
    6. Carson, Richard T. & McCubbin, Donald R., 1998. "Policy Paper 32: Emissions and Development in the United States: International Implications," Institute on Global Conflict and Cooperation, Working Paper Series qt02t32857, Institute on Global Conflict and Cooperation, University of California.
    7. King, Carey W., 2020. "An integrated biophysical and economic modeling framework for long-term sustainability analysis: the HARMONEY model," Ecological Economics, Elsevier, vol. 169(C).
    8. Vicknair, David & Tansey, Michael & O'Brien, Thomas E., 2022. "Measuring fossil fuel reserves: A simulation and review of the U.S. Securities and Exchange Commission approach," Resources Policy, Elsevier, vol. 79(C).
    9. Pogany, Peter, 2013. "Thermodynamic Isolation and the New World Order," MPRA Paper 49924, University Library of Munich, Germany.
    10. Smulders, Sjak & Gradus, Raymond, 1996. "Pollution abatement and long-term growth," European Journal of Political Economy, Elsevier, vol. 12(3), pages 505-532, November.
    11. Badunenko, Oleg & Galeotti, Marzio & Hunt, Lester C., 2021. "Better to grow or better to improve? Measuring environmental efficiency in OECD countries with a Stochastic Environmental Kuznets Frontier," FEEM Working Papers 316226, Fondazione Eni Enrico Mattei (FEEM).
    12. Khalid Saeed, 2014. "Jay Forrester's operational approach to economics," System Dynamics Review, System Dynamics Society, vol. 30(4), pages 233-261, October.
    13. William Tracy, 2014. "Paradox Lost: The Evolution of Strategies in Selten’s Chain Store Game," Computational Economics, Springer;Society for Computational Economics, vol. 43(1), pages 83-103, January.
    14. Kollman, Ken & Miller, John H. & Page, Scott E., 1997. "Landscape formation in a spatial voting model," Economics Letters, Elsevier, vol. 55(1), pages 121-130, August.
    15. Ondřej Šíma, 2020. "Reálná ekonomika jako zdroj nerovnováhy obchodní bilance - základní přístup [Real Economy as a Source of Trade Balance Disequilibrium - Basic Approach]," Politická ekonomie, Prague University of Economics and Business, vol. 2020(3), pages 322-347.
    16. Robbie Maris & Mark Holmes, 2023. "Economic Growth Theory and Natural Resource Constraints: A Stocktake and Critical Assessment," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 56(2), pages 255-268, June.
    17. Guido A Veldhuis & Nico M de Reus & Bas MJ Keijser, 2020. "Concept development for comprehensive operations support with modeling and simulation," The Journal of Defense Modeling and Simulation, , vol. 17(1), pages 99-116, January.
    18. Joshua M. Epstein, 2007. "Agent-Based Computational Models and Generative Social Science," Introductory Chapters, in: Generative Social Science Studies in Agent-Based Computational Modeling, Princeton University Press.
    19. Cuthbertson, Madison & Workman, Mark & Brophy, Aoife, 2024. "Without mandated demand for greenhouse gas removal – High integrity GtCO2-scale global deployment will be jeopardized: Insight from US economic policy 2020–23," Applied Energy, Elsevier, vol. 372(C).
    20. Richard T. Carson, 2010. "The Environmental Kuznets Curve: Seeking Empirical Regularity and Theoretical Structure," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 4(1), pages 3-23, Winter.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:44:y:1998:i:6:p:820-830. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.