A fuzzy set approach to assess the predictive accuracy of land use simulations

The predictive accuracy of land use models is frequently assessed by comparing two data sets: the simulated land use map and the observed land use map at the end of the simulation period. A common statistic for this is Kappa, which expresses the agreement between two categorical maps, corrected for the agreement as can be expected by chance. This chance agreement is based on a stochastic model of random allocation given the distribution of class sizes. Two existing statistics extend Kappa to make it more appropriate for the assessment of land use models: Fuzzy Kappa uses fuzzy set theory to include degrees of similarity, which adds geographical nuance because it distinguishes between small and large disagreement in position and in land use classes. Kappa Simulation, on the other hand, addresses the stochastic model that underlies the expected agreement: when a model starts from an initial land use map and subsequently makes changes to it, a stochastic model of random allocation given the distribution of class sizes has little relevance. The expected accuracy in Kappa Simulation is therefore based on transition probabilities relative to the initial map. This paper presents Fuzzy Kappa Simulation, a statistic that combines the geographical nuance of Fuzzy Kappa with the stochastic model of Kappa Simulation. This new statistic is demonstrated on a case study example and results are compared with other variations of Kappa. The comparison confirms that Fuzzy Kappa Simulation is the only statistic to evaluate models in terms of land use transitions, while also being sensitive to geographical nuance.