IDEAS home Printed from https://ideas.repec.org/a/jas/jasssj/2014-43-2.html
   My bibliography  Save this article

A New Method to Evaluate Simulation Models: The Calibration Profile (CP) Algorithm

Author

Listed:
  • Romain Reuillon
  • Clara Schmitt
  • Ricardo De Aldama
  • Jean-Baptiste Mouret

Abstract

Models of social systems generally contain free parameters that cannot be evaluated directly from data. A calibration phase is therefore necessary to assess the capacity of the model to produce the expected dynamics. However, despite the high computational cost of this calibration it doesn't produce a global picture of the relationship between the parameter space and the behaviour space of the model. The Calibration Profile (CP) algorithm is an innovative method extending the concept of automated calibration processes. It computes a profile that depicts the effect of each single parameter on the model behaviour, independently from the others. A 2-dimensional graph is thus produced exposing the impact of the parameter under study on the capacity of the model to produce expected dynamics. The first part of this paper is devoted to the formal description of the CP algorithm. In the second part,we apply it to an agent based geographical model (SimpopLocal). The analysis of the results brings to light novel insights on the model.

Suggested Citation

  • Romain Reuillon & Clara Schmitt & Ricardo De Aldama & Jean-Baptiste Mouret, 2015. "A New Method to Evaluate Simulation Models: The Calibration Profile (CP) Algorithm," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(1), pages 1-12.
    • Handle: RePEc:jas:jasssj:2014-43-2
    as

    Download full text from publisher

    File URL: https://www.jasss.org/18/1/12/12.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. A G Wilson, 1971. "A Family of Spatial Interaction Models, and Associated Developments," Environment and Planning A, , vol. 3(1), pages 1-32, March.
    2. Brian J. L. Berry, 1964. "Cities As Systems Within Systems Of Cities," Papers in Regional Science, Wiley Blackwell, vol. 13(1), pages 147-163, January.
    3. Anne Bretagnolle & Denise Pumain, 2010. "Simulating Urban Networks through Multiscalar Space-Time Dynamics: Europe and the United States, 17th-20th Centuries," Urban Studies, Urban Studies Journal Limited, vol. 47(13), pages 2819-2839, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Antoine Peris & Evert Meijers & Maarten Ham, 2018. "The Evolution of the Systems of Cities Literature Since 1995: Schools of Thought and their Interaction," Networks and Spatial Economics, Springer, vol. 18(3), pages 533-554, September.
    2. Daniel A. Griffith & Manfred M. Fischer, 2016. "Constrained Variants of the Gravity Model and Spatial Dependence: Model Specification and Estimation Issues," Advances in Spatial Science, in: Roberto Patuelli & Giuseppe Arbia (ed.), Spatial Econometric Interaction Modelling, chapter 0, pages 37-66, Springer.
    3. M. Centorrino & F. Ofria & D. Farinella, 2010. "Convergence and divergence processes between the Mezzogiorno and the Centre- North ten years after the adoption of the EMU," Rivista economica del Mezzogiorno, Società editrice il Mulino, issue 3, pages 453-486.
    4. Peter Wissoker & Desiree Fields & Rachel Weber & Elvin Wyly, 2014. "Rethinking Real Estate Finance in the Wake of a Boom: A Celebration of the Twentieth Anniversary of the Publication of the Double Issue on Property and Finance in Environment and Planning A," Environment and Planning A, , vol. 46(12), pages 2787-2794, December.
    5. Xu, Gang & Xu, Zhibang & Gu, Yanyan & Lei, Weiqian & Pan, Yupiao & Liu, Jie & Jiao, Limin, 2020. "Scaling laws in intra-urban systems and over time at the district level in Shanghai, China," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 560(C).
    6. Karima Kourtit & Peter Nijkamp, 2013. "The Use of Visual Decision Support Tools in an Interactive Stakeholder Analysis—Old Ports as New Magnets for Creative Urban Development," Sustainability, MDPI, vol. 5(10), pages 1-27, October.
    7. Halás, Marián & Klapka, Pavel & Kladivo, Petr, 2014. "Distance-decay functions for daily travel-to-work flows," Journal of Transport Geography, Elsevier, vol. 35(C), pages 107-119.
    8. Qingsong He & Lingping Huang & Jing Li, 2022. "Rediscovering the Scaling Law of Urban Land from a Multi-Scale Perspective—A Case Study of Wuhan," Land, MDPI, vol. 11(6), pages 1-15, June.
    9. Bade, Franz-Josef & Niebuhr, Annekatrin & Schönert, Matthias, 2000. "Spatial structural change – Evidence and prospects," HWWA Discussion Papers 87, Hamburg Institute of International Economics (HWWA).
    10. Ahmed El-Geneidy & David Levinson, 2011. "Place Rank: Valuing Spatial Interactions," Networks and Spatial Economics, Springer, vol. 11(4), pages 643-659, December.
    11. Nijkamp, Peter & Poot, Jacques, 1987. "Dynamics of generalised spatial interaction models," Regional Science and Urban Economics, Elsevier, vol. 17(3), pages 367-390, August.
    12. Nancy Ruiz Estupiñán & Carlos Marmolejo Duarte & Moira Tornés Fernández, 2013. "Functional Polycentricity And Its Role In The Emergence Of Structural Places. The Case Of Major Spanish Metropolitan Areas," ERSA conference papers ersa13p634, European Regional Science Association.
    13. Tiller, Kara Carroll & Thill, Jean-Claude, 2017. "Spatial patterns of landside trade impedance in containerized South American exports," Journal of Transport Geography, Elsevier, vol. 58(C), pages 272-285.
    14. D. F. Darwent, 1969. "Growth Poles and Growth Centers in Regional Planning—A Review," Environment and Planning A, , vol. 1(1), pages 5-31, June.
    15. Denise Pumain & Céline Rozenblat, 2019. "Two metropolisation gradients in the European system of cities revealed by scaling laws," Environment and Planning B, , vol. 46(9), pages 1645-1662, November.
    16. Aurélie Lalanne, 2014. "Zipf’s Law and Canadian Urban Growth," Urban Studies, Urban Studies Journal Limited, vol. 51(8), pages 1725-1740, June.
    17. Evert Meijers & Martijn Burger & Michiel Meeteren & Zachary Neal & Ben Derudder, 2016. "Disentangling agglomeration and network externalities: A conceptual typology," Papers in Regional Science, Wiley Blackwell, vol. 95(1), pages 61-80, March.
    18. Chen, Yanguang, 2017. "Multi-scaling allometric analysis for urban and regional development," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 465(C), pages 673-689.
    19. Jo Ann Lane & Barry Boehm, 2008. "System of systems lead system integrators: Where Do they spend their time and what makes them more or less efficient?," Systems Engineering, John Wiley & Sons, vol. 11(1), pages 81-91, March.
    20. Robert Cromley & Dean Hanink, 2008. "Population growth and the development of a central place system," Journal of Geographical Systems, Springer, vol. 10(4), pages 383-405, December.

    More about this item

    Keywords

    Calibration Profile; Model Evaluation;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jas:jasssj:2014-43-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Francesco Renzini (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.