IDEAS home Printed from https://ideas.repec.org/a/eee/ecomod/v222y2011i8p1367-1375.html
   My bibliography  Save this article

Revisiting Kappa to account for change in the accuracy assessment of land-use change models

Author

Listed:
  • van Vliet, Jasper
  • Bregt, Arnold K.
  • Hagen-Zanker, Alex

Abstract

Land-use change models are typically calibrated to reproduce known historic changes. Calibration results can then be assessed by comparing two datasets: the simulated land-use map and the actual land-use map at the same time. A common method for this is the Kappa statistic, which expresses the agreement between two categorical datasets corrected for the expected agreement. This expected agreement is based on a stochastic model of random allocation given the distribution of class sizes. However, when a model starts from an initial land-use map and makes changes to it, that stochastic model does not pose a meaningful reference level. This paper introduces KSimulation, a statistic that is identical in form to the Kappa statistic but instead applies a more appropriate stochastic model of random allocation of class transitions relative to the initial map. The new method is illustrated on a simple example and then the results of the Kappa statistic and KSimulation are compared using the results of a land-use model. It is found that only KSimulation truly tests models in their capacity to explain land-use changes over time, and unlike Kappa it does not inflate results for simulations where little change takes place over time.

Suggested Citation

  • van Vliet, Jasper & Bregt, Arnold K. & Hagen-Zanker, Alex, 2011. "Revisiting Kappa to account for change in the accuracy assessment of land-use change models," Ecological Modelling, Elsevier, vol. 222(8), pages 1367-1375.
  • Handle: RePEc:eee:ecomod:v:222:y:2011:i:8:p:1367-1375
    DOI: 10.1016/j.ecolmodel.2011.01.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304380011000494
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ecolmodel.2011.01.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rutherford, Gillian N. & Bebi, Peter & Edwards, Peter J. & Zimmermann, Niklaus E., 2008. "Assessing land-use statistics to model land cover change in a mountainous landscape in the European Alps," Ecological Modelling, Elsevier, vol. 212(3), pages 460-471.
    2. Roger White, 2006. "Pattern based map comparisons," Journal of Geographical Systems, Springer, vol. 8(2), pages 145-164, July.
    3. Steven M Manson, 2007. "Challenges in Evaluating Models of Geographic Complexity," Environment and Planning B, , vol. 34(2), pages 245-260, April.
    4. Robert Pontius & Wideke Boersma & Jean-Christophe Castella & Keith Clarke & Ton Nijs & Charles Dietzel & Zengqiang Duan & Eric Fotsing & Noah Goldstein & Kasper Kok & Eric Koomen & Christopher Lippitt, 2008. "Comparing the input, output, and validation maps for several models of land change," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 42(1), pages 11-37, March.
    5. van Delden, Hedwig & Stuczynski, Tomasz & Ciaian, Pavel & Paracchini, Maria Luisa & Hurkens, Jelle & Lopatka, Artur & Shi, Yu-e & Prieto, Oscar Gomez & Calvo, Silvia & van Vliet, Jasper & Vanhout, Roe, 2010. "Integrated assessment of agricultural policies with dynamic land use change modelling," Ecological Modelling, Elsevier, vol. 221(18), pages 2153-2166.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jaekyung Lee & Galen Newman & Yunmi Park, 2018. "A Comparison of Vacancy Dynamics between Growing and Shrinking Cities Using the Land Transformation Model," Sustainability, MDPI, vol. 10(5), pages 1-17, May.
    2. Bo Sun & Derek T. Robinson, 2018. "Comparison of Statistical Approaches for Modelling Land-Use Change," Land, MDPI, vol. 7(4), pages 1-33, November.
    3. Yang, Yuanyuan & Bao, Wenkai & Liu, Yansui, 2020. "Scenario simulation of land system change in the Beijing-Tianjin-Hebei region," Land Use Policy, Elsevier, vol. 96(C).
    4. Aritta Suwarno & Meine van Noordwijk & Hans-Peter Weikard & Desi Suyamto, 2018. "Indonesia’s forest conversion moratorium assessed with an agent-based model of Land-Use Change and Ecosystem Services (LUCES)," Mitigation and Adaptation Strategies for Global Change, Springer, vol. 23(2), pages 211-229, February.
    5. Gärtner, Dominique & Keller, Armin & Schulin, Rainer, 2013. "A simple regional downscaling approach for spatially distributing land use types for agricultural land," Agricultural Systems, Elsevier, vol. 120(C), pages 10-19.
    6. Hong Shi & Ji Yang & Qijuan Liu & Taohong Li & Ning Chris Chen, 2024. "Impacts of Climate and Land-Use Change on Fraction Vegetation Coverage Based on PLUS-Dimidiate Pixel Model," Sustainability, MDPI, vol. 16(23), pages 1-18, November.
    7. Ju-Sung Lee & Tatiana Filatova & Arika Ligmann-Zielinska & Behrooz Hassani-Mahmooei & Forrest Stonedahl & Iris Lorscheid & Alexey Voinov & J. Gareth Polhill & Zhanli Sun & Dawn C. Parker, 2015. "The Complexities of Agent-Based Modeling Output Analysis," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(4), pages 1-4.
    8. Yipeng Zhang & Yunbing Gao & Bingbo Gao & Yuchun Pan & Mingyang Yan, 2015. "An Efficient Graph-based Method for Long-term Land-use Change Statistics," Sustainability, MDPI, vol. 8(1), pages 1-14, December.
    9. Fondevilla, Cristian & Àngels Colomer, M. & Fillat, Federico & Tappeiner, Ulrike, 2016. "Using a new PDP modelling approach for land-use and land-cover change predictions: A case study in the Stubai Valley (Central Alps)," Ecological Modelling, Elsevier, vol. 322(C), pages 101-114.
    10. Margaret Gitau & Nathaniel Bailey, 2012. "Multi-Layer Assessment of Land Use and Related Changes for Decision Support in a Coastal Zone Watershed," Land, MDPI, vol. 1(1), pages 1-27, December.
    11. Xiaoli Hu & Xin Li & Ling Lu, 2018. "Modeling the Land Use Change in an Arid Oasis Constrained by Water Resources and Environmental Policy Change Using Cellular Automata Models," Sustainability, MDPI, vol. 10(8), pages 1-14, August.
    12. Charlotte Shade & Peleg Kremer, 2019. "Predicting Land Use Changes in Philadelphia Following Green Infrastructure Policies," Land, MDPI, vol. 8(2), pages 1-19, February.
    13. Wu, Wei & Yeager, Kevin M. & Peterson, Mark S. & Fulford, Richard S., 2015. "Neutral models as a way to evaluate the Sea Level Affecting Marshes Model (SLAMM)," Ecological Modelling, Elsevier, vol. 303(C), pages 55-69.
    14. Prashanti Sharma & Rajesh Bahadur Thapa & Mir Abdul Matin, 2020. "Examining forest cover change and deforestation drivers in Taunggyi District, Shan State, Myanmar," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 22(6), pages 5521-5538, August.
    15. Cláudia M. Viana & Jorge Rocha, 2020. "Evaluating Dominant Land Use/Land Cover Changes and Predicting Future Scenario in a Rural Region Using a Memoryless Stochastic Method," Sustainability, MDPI, vol. 12(10), pages 1-28, May.
    16. Javier Muro & Leo Zurita-Arthos & José Jara & Esteban Calderón & Richard Resl & Andreas Rienow & Valerie Graw, 2020. "Earth Observation for Settlement Mapping of Amazonian Indigenous Populations to Support SDG7," Resources, MDPI, vol. 9(8), pages 1-17, August.
    17. van Vliet, Jasper & Hagen-Zanker, Alex & Hurkens, Jelle & van Delden, Hedwig, 2013. "A fuzzy set approach to assess the predictive accuracy of land use simulations," Ecological Modelling, Elsevier, vol. 261, pages 32-42.
    18. Ding, Dan & Liu, Xiaoping & Xu, Xiaocong, 2024. "Projecting the future fine-resolution carbon dioxide emissions under the shared socioeconomic pathways for carbon peak evaluation," Applied Energy, Elsevier, vol. 365(C).
    19. Ramita Manandhar & Inakwu O.A. Odeh & Tihomir Ancev, 2014. "Assessment of Spatial-Temporal Expansion of Built-up and Residential-Commercial Dwellings with Some Economic Implications: A Case Study in the Lower Hunter of Eastern Australia," Land, MDPI, vol. 3(1), pages 1-21, March.
    20. Gustavo Larrea‐Gallegos & Ian Vázquez‐Rowe, 2022. "Exploring machine learning techniques to predict deforestation to enhance the decision‐making of road construction projects," Journal of Industrial Ecology, Yale University, vol. 26(1), pages 225-239, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecomod:v:222:y:2011:i:8:p:1367-1375. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/ecological-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.