IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v55y2021i5d10.1007_s11135-020-01073-8.html
   My bibliography  Save this article

Mapping validity and validation in modelling for interdisciplinary research

Author

Listed:
  • Guus ten Broeke

    (Wageningen University & Research)

  • Hilde Tobi

    (Wageningen University & Research)

Abstract

Complex Adaptive Systems (CAS) is an interdisciplinary and dynamic modelling approach for the study of today’s global challenges. It is used for the explanation, description, and prediction of behaviours of system components and the system at large. To understand and assess the quality of research in which CAS models are designed and used, a thorough understanding of the meanings of ‘validity’ from social science research methodology and ‘validation’ from simulation modelling is needed. In this paper, we first describe the modelling process. Then, we analyse the concepts ‘validity’ and ‘validation’ as used in a set of research methodology textbooks and a set of modelling textbooks. We present one single map that integrates validity as characteristic of the model input, the modelling process, model validation, and the validity of the model built. The map is illustrated by means of one example. The terminology proposed in the map allows to describe and distinguish between the validity of primary research used for input in the model, how the quality of the modelling depends on structural and behavioural validation, and, how the assessment of the validity of the model is informed by these types of validation plus research with independent data.

Suggested Citation

  • Guus ten Broeke & Hilde Tobi, 2021. "Mapping validity and validation in modelling for interdisciplinary research," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(5), pages 1613-1630, October.
    • Handle: RePEc:spr:qualqt:v:55:y:2021:i:5:d:10.1007_s11135-020-01073-8
    DOI: 10.1007/s11135-020-01073-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-020-01073-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-020-01073-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Paul Windrum & Giorgio Fagiolo & Alessio Moneta, 2007. "Empirical Validation of Agent-Based Models: Alternatives and Prospects," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 10(2), pages 1-8.
    2. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    3. Augusiak, Jacqueline & Van den Brink, Paul J. & Grimm, Volker, 2014. "Merging validation and evaluation of ecological models to ‘evaludation’: A review of terminology and a practical approach," Ecological Modelling, Elsevier, vol. 280(C), pages 117-128.
    4. Marco Fattore & Rosanna Grassi, 2015. "Qualitative and quantitative methods in complex socio-economic systems," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(4), pages 1549-1551, July.
    5. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    6. Grimm, Volker & Augusiak, Jacqueline & Focks, Andreas & Frank, Béatrice M. & Gabsi, Faten & Johnston, Alice S.A. & Liu, Chun & Martin, Benjamin T. & Meli, Mattia & Radchuk, Viktoriia & Thorbek, Pernil, 2014. "Towards better modelling and decision support: Documenting model development, testing, and analysis using TRACE," Ecological Modelling, Elsevier, vol. 280(C), pages 129-139.
    7. Chih-Tung Hsiao & Hui-Ling Peng & Hui-Hua Huang, 2012. "The impact of sports ethics of professional baseball player to the development of sports industry in Taiwan," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(6), pages 1753-1767, October.
    8. Hilde Tobi & Jarl K. Kampen, 2018. "Research design: the methodology for interdisciplinary research framework," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(3), pages 1209-1225, May.
    9. Bruce Edmonds & Christophe Le Page & Mike Bithell & Edmund Chattoe-Brown & Volker Grimm & Ruth Meyer & Cristina Montañola-Sales & Paul Ormerod & Hilton Root & Flaminio Squazzoni, 2019. "Different Modelling Purposes," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 22(3), pages 1-6.
    10. Jule Thober & Birgit Müller & Jürgen Groeneveld & Volker Grimm, 2017. "Agent-Based Modelling of Social-Ecological Systems: Achievements, Challenges, and a Way Forward," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(2), pages 1-8.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Troost, Christian & Huber, Robert & Bell, Andrew R. & van Delden, Hedwig & Filatova, Tatiana & Le, Quang Bao & Lippe, Melvin & Niamir, Leila & Polhill, J. Gareth & Sun, Zhanli & Berger, Thomas, 2023. "How to keep it adequate: A protocol for ensuring validity in agent-based simulation," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 159, pages 1-21.
    2. Lamperti, Francesco & Bosetti, Valentina & Roventini, Andrea & Tavoni, Massimo & Treibich, Tania, 2021. "Three green financial policies to address climate risks," Journal of Financial Stability, Elsevier, vol. 54(C).
    3. Francesco Lamperti & Giovanni Dosi & Mauro Napoletano & Andrea Roventini & Alessandro Sapio, 2018. "And then he wasn't a she : Climate change and green transitions in an agent-based integrated assessment model," Working Papers hal-03443464, HAL.
    4. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    5. Gianluca Capone & Franco Malerba & Richard R. Nelson & Luigi Orsenigo & Sidney G. Winter, 2019. "History friendly models: retrospective and future perspectives," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 9(1), pages 1-23, March.
    6. Dosi, Giovanni & Palagi, Elisa & Roventini, Andrea & Russo, Emanuele, 2023. "Do patents really foster innovation in the pharmaceutical sector? Results from an evolutionary, agent-based model," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 564-589.
    7. Lamperti, F. & Dosi, G. & Napoletano, M. & Roventini, A. & Sapio, A., 2018. "Faraway, So Close: Coupled Climate and Economic Dynamics in an Agent-based Integrated Assessment Model," Ecological Economics, Elsevier, vol. 150(C), pages 315-339.
    8. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    9. Lamperti, F. & Dosi, G. & Napoletano, M. & Roventini, A. & Sapio, A., 2020. "Climate change and green transitions in an agent-based integrated assessment model," Technological Forecasting and Social Change, Elsevier, vol. 153(C).
    10. repec:spo:wpmain:info:hdl:2441/401t6job098n79ch91o9giov9d is not listed on IDEAS
    11. Giovanni Dosi & Andrea Roventini, 2019. "More is different ... and complex! the case for agent-based macroeconomics," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 1-37, March.
    12. Dosi, Giovanni & Lamperti, Francesco & Mazzucato, Mariana & Napoletano, Mauro & Roventini, Andrea, 2023. "Mission-oriented policies and the “Entrepreneurial State” at work: An agent-based exploration," Journal of Economic Dynamics and Control, Elsevier, vol. 151(C).
    13. An, Li & Grimm, Volker & Sullivan, Abigail & Turner II, B.L. & Malleson, Nicolas & Heppenstall, Alison & Vincenot, Christian & Robinson, Derek & Ye, Xinyue & Liu, Jianguo & Lindkvist, Emilie & Tang, W, 2021. "Challenges, tasks, and opportunities in modeling agent-based complex systems," Ecological Modelling, Elsevier, vol. 457(C).
    14. Vandin, Andrea & Giachini, Daniele & Lamperti, Francesco & Chiaromonte, Francesca, 2022. "Automated and distributed statistical analysis of economic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    15. Seri, Raffaello & Martinoli, Mario & Secchi, Davide & Centorrino, Samuele, 2021. "Model calibration and validation via confidence sets," Econometrics and Statistics, Elsevier, vol. 20(C), pages 62-86.
    16. repec:hal:spmain:info:hdl:2441/401t6job098n79ch91o9giov9d is not listed on IDEAS
    17. Mario Martinoli & Alessio Moneta & Gianluca Pallante, 2022. "Calibration and Validation of Macroeconomic Simulation Models by Statistical Causal Search," LEM Papers Series 2022/33, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    18. Lapp, Maya & Long, Colby, 2022. "A new approach to agent-based models of Community Resource Management based on the analysis of cheating, monitoring, and sanctioning," Ecological Modelling, Elsevier, vol. 468(C).
    19. Planque, Benjamin & Aarflot, Johanna M. & Buttay, Lucie & Carroll, JoLynn & Fransner, Filippa & Hansen, Cecilie & Husson, Bérengère & Langangen, Øystein & Lindstrøm, Ulf & Pedersen, Torstein & Primice, 2022. "A standard protocol for describing the evaluation of ecological models," Ecological Modelling, Elsevier, vol. 471(C).
    20. Carrella, Ernesto & Saul, Steven & Marshall, Kristin & Burgess, Matthew G. & Cabral, Reniel B. & Bailey, Richard M. & Dorsett, Chris & Drexler, Michael & Madsen, Jens Koed & Merkl, Andreas, 2020. "Simple Adaptive Rules Describe Fishing Behaviour Better than Perfect Rationality in the US West Coast Groundfish Fishery," Ecological Economics, Elsevier, vol. 169(C).
    21. Alperen Bektas & Valentino Piana & René Schumann, 2021. "A meso-level empirical validation approach for agent-based computational economic models drawing on micro-data: a use case with a mobility mode-choice model," SN Business & Economics, Springer, vol. 1(6), pages 1-25, June.
    22. Kukacka, Jiri & Jang, Tae-Seok & Sacht, Stephen, 2018. "On the estimation of behavioral macroeconomic models via simulated maximum likelihood," Economics Working Papers 2018-11, Christian-Albrechts-University of Kiel, Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:55:y:2021:i:5:d:10.1007_s11135-020-01073-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.